Quantcast

osgearth_package: handling of transparency in TIFF files

classic Classic list List threaded Threaded
19 messages Options
rammann rammann
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

osgearth_package: handling of transparency in TIFF files

I have TIFF files that contain some transparent parts. When creating a TMS repo then these transparent tiles are all exported as well. Since there's a lot of data this results in losts of "wasted" disk space. Is there a chance you could filter out transparent tiles in osgearth_package?

Enclosed is a ZIP (1MB) which contains an example TIFF and earth file.

osgearth_packaged test.earth --tms --ext png --out output --max-level 20

test.zip
gwaldron gwaldron
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: osgearth_package: handling of transparency in TIFF files

rammann,

It was on the list. Grab the latest and try. You can defeat the feature (and write out "empty" tiles) with the --keep-empties flag.

Glenn Waldron / Pelican Mapping / @glennwaldron
rammann rammann
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: osgearth_package: handling of transparency in TIFF files

thanks. It seems to work on most files, however on some TIFFS the skipping will stop the whole job...

[osgearth_package] Packaging image layer "test"
[TMSPackager] MIME-TYPE = image/png, Extension = png
[TMSPackager] Wrote tile 0/0/0 (SW=-180,-90 NE=0,90, SRS=WGS 84)
[TMSPackager] Skipping empty tile 1/1/0

In this case it only wrote one tile and then quits.

I have managed to create a small TIFF+earth file for reproducing the issue, see test.zip (~4mb)

Like usual the command line I've been using:
osgearth_package --tms test.earth --ext png --out output --max-level 21

Thanks,
Ronald
gwaldron gwaldron
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: osgearth_package: handling of transparency in TIFF files

Ronald,

Thanks for the test data.

Unfortunately, I'm not able to duplicate the problem you're having. I ran the packager on your earth file using the command line you provided. It skipped no tiles, generated 1600+ tiles, and the output looks correct.

First few lines of the log are:

[osgEarth]  [TMSPackager] Wrote tile 0/0/0 (SW=-180,-90 NE=0,90, SRS=WGS 84)
[osgEarth]  [TMSPackager] Wrote tile 1/1/0 (SW=-90,0 NE=0,90, SRS=WGS 84)
[osgEarth]  [TMSPackager] Wrote tile 2/2/1 (SW=-90,0 NE=-45,45, SRS=WGS 84)
[osgEarth]  [TMSPackager] Wrote tile 3/4/2 (SW=-90,22.5 NE=-67.5,45, SRS=WGS 84)
[osgEarth]  [TMSPackager] Wrote tile 4/9/4 (SW=-78.75,33.75 NE=-67.5,45, SRS=WGS 84)
[osgEarth]  [TMSPackager] Wrote tile 5/18/9 (SW=-78.75,33.75 NE=-73.125,39.375, SRS=WGS 84)
[osgEarth]  [TMSPackager] Wrote tile 6/36/18 (SW=-78.75,36.5625 NE=-75.9375,39.375, SRS=WGS 84)
...
and so on.

Glenn Waldron / Pelican Mapping / @glennwaldron
rammann rammann
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: osgearth_package: handling of transparency in TIFF files

Glenn,

here's a more detailed log:

[osgEarth]  [Map] Map profile is: [srs=WGS 84, min=-180,-90 max=180,90 lod0=2,1 vdatum=geodetic]
[osgEarth]  [TextureCompositor] Compositing technique = MULTITEXTURE/GPU
[osgEarth]  [TextureCompositor] Allocated SLOT 0; primary slot for layer "test"
[osgEarth]  [OverlayDecorator] Using texture size = 4096
[osgEarth]  [osgearth_package] Packaging image layer "test"
[osgEarth]  [ImageLayer] Layer "test" create image for "0/0/0"
[osgEarth]  [TerrainLayer] Initializing tile source ...
[osgEarth]  [GDAL driver] Driver found 1 files:
[osgEarth]  [GDAL driver] C:\temp\tms\test.tif
[osgEarth]  [GDAL driver] Geo extents: -77.219796, 38.738466 => -76.821581, 39.080156
[osgEarth]  [GDAL driver] Resolution= 7.6905168e-05x7.6905168e-05 max=7.6905168e-05
[osgEarth]  [GDAL driver] Max Data Level: 14
[osgEarth]  [GDAL driver] Data Extents: SW=-77.2197959468,38.7384659417 NE=-76.8215809887,39.0801556016, SRS=WGS 84
[osgEarth]  [GDAL driver] ReadWindow 5178x4443 DestWindow 1x1
[osgEarth]  [ImageLayer] test : 0/0/0 result ok
[osgEarth]  [TMSPackager] MIME-TYPE = image/png, Extension = png
[osgEarth]  [TMSPackager] Tile 0/0/0 already exists
[osgEarth]  [ImageLayer] Layer "test" create image for "1/0/0"
[osgEarth]  [ImageLayer] createImageFromTileSource: hasDataInExtent(1/0/0) == false
[osgEarth]  [ImageLayer] test : 1/0/0 result invalid
[osgEarth]  [ImageLayer] Layer "test" create image for "1/1/0"
[osgEarth]  [GDAL driver] ReadWindow 5178x4443 DestWindow 2x1
[osgEarth]  [ImageLayer] test : 1/1/0 result ok
[osgEarth]  [TMSPackager] Skipping empty tile 1/1/0
[osgEarth]  [ImageLayer] Layer "test" create image for "1/0/1"
[osgEarth]  [ImageLayer] createImageFromTileSource: hasDataInExtent(1/0/1) == false
[osgEarth]  [ImageLayer] test : 1/0/1 result invalid
[osgEarth]  [ImageLayer] Layer "test" create image for "1/1/1"
[osgEarth]  [ImageLayer] createImageFromTileSource: hasDataInExtent(1/1/1) == false
[osgEarth]  [ImageLayer] test : 1/1/1 result invalid
[osgEarth]  [ImageLayer] Layer "test" create image for "0/1/0"
[osgEarth]  [ImageLayer] createImageFromTileSource: hasDataInExtent(0/1/0) == false
[osgEarth]  [ImageLayer] test : 0/1/0 result invalid
[osgEarth]  [Map] Map profile is: [srs=WGS 84, min=-180,-90 max=180,90 lod0=2,1 vdatum=geodetic]
[osgEarth]  [TextureCompositor] Compositing technique = MULTITEXTURE/GPU
[osgEarth]  [TextureCompositor] Allocated SLOT 0; primary slot for layer "test"
[osgEarth]  [OverlayDecorator] Using texture size = 4096
[osgEarth]  [osgearth_package] Wrote earth file to "output\tms_test.earth"
[osgEarth]  ~Map
[osgEarth]  ~Map

Any ideas where I could look into to solve this issue?

Strange that you can't reproduce it. I used the source from trunk to make sure everything is up-to-date but the problem still doesn't go away...

Ronald
gwaldron gwaldron
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: osgearth_package: handling of transparency in TIFF files

Ronald,
There's an alpha threshold that's used to decide whether a tile is "empty". Try adjusting it and see if that makes a difference.

In TMSPackager.cpp, line 101, change it to:

  ...isEmptyImage(image.getImage(), 0.0f)...

Let me know if that works.

Glenn Waldron / Pelican Mapping / @glennwaldron
rammann rammann
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: osgearth_package: handling of transparency in TIFF files

Glenn,
sorry this doesn't really work for me.

I did another test and added the --keep-empties parameter. The result is 1626 tiles including the transparent ones. Looking at the files I see that 1/1/1 png is a transparent tile. This is the one that makes osgearth_packager stop. The next tile that is not transparent is 3/4/5. So I'm wondering why I don't see a non-transparent tile at level 1... the 1/1/0 that I see in your log is not existing in my output.

Ronald
gwaldron gwaldron
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: osgearth_package: handling of transparency in TIFF files

Ronald,
The "y" index is inverted, for osgEarth's TileKey (what appears in the log) versus the TMS filename. So, TileKey "1/1/0" corresponds to the file "1/1/1.png". Confusing I know, perhaps it would be better if the log printing the actual filename.

Still, that doesn't account for your issue. My 1/1/1.png is not transparent -- it's attached.

1.png


Glenn Waldron / Pelican Mapping / @glennwaldron
rammann rammann
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: osgearth_package: handling of transparency in TIFF files

What I think is little weird is that you said osgearth_package didn't skip any tiles. But there's plenty of transparency in the TIFF so it should really skip some....
rammann rammann
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: osgearth_package: handling of transparency in TIFF files

Glenn,

after some more debugging I found out that createImage in the GDAL driver is creating a transparent image for 1/1/1. I'm using GDAL 1.9.0 stable - are you maybe using a different version?
gwaldron gwaldron
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: osgearth_package: handling of transparency in TIFF files

I'm using GDAL 1.6.

I would recommend building against 1.6 and see if the problem goes away. If so, then we know where to start looking.
Glenn Waldron / Pelican Mapping / @glennwaldron
rammann rammann
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: osgearth_package: handling of transparency in TIFF files

I still had GDAL 1.7.1 somewhere.... just tested and with this version osgearth_package works fine!
remoe remoe
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: osgearth_package: handling of transparency in TIFF files

i use GDAL 1.8.1 and it works fine.
Remo Eichenberger,
gwaldron gwaldron
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: osgearth_package: handling of transparency in TIFF files

Wow. So just 1.9 causes the issue? I'll open up a ticket so we have that on the list to investigate.

Thanks for doing the tests, very helpful.

Glenn Waldron / Pelican Mapping / @glennwaldron
jasonbeverage jasonbeverage
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: osgearth_package: handling of transparency in TIFF files

Are you running with a cache path set?  Try clearing your cache or running the packager with the OSGEARTH_NO_CACHE=1 env var set.  I think you're picking up files from an old cache where you had another image named test.tif.

I just tested with a clean cache against GDAL 1.9 and your example worked just fine, I'm guessing when you switched to GDAL 1.7 you also changed your cache path and it started working correctly.

Jason

On Fri, Apr 27, 2012 at 8:11 AM, gwaldron [via osgEarth] <[hidden email]> wrote:
Wow. So just 1.9 causes the issue? I'll open up a ticket so we have that on the list to investigate.

Thanks for doing the tests, very helpful.

Glenn Waldron / @glennwaldron



If you reply to this email, your message will be added to the discussion below:
http://forum.osgearth.org/osgearth-package-handling-of-transparency-in-TIFF-files-tp7483503p7505984.html
To start a new topic under osgEarth, email [hidden email]
To unsubscribe from osgEarth, click here.
NAML

Andreas Roth Andreas Roth
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: osgearth_package: handling of transparency in TIFF files

Hi,

in the .earth file posted by rammann no cache is specified, so the osgearth does not cache any tiles.
<?xml version="1.0"?> 
<map version="2"> 
	<image name="test" driver="gdal"> 
		<url>test.tif</url> 
	</image> 
</map> 

@jasonbeverage: Have you tested the osgearth_packager with the provided TIFF-file and .earth file with GDAL 1.9 without any trouble?

Andreas

jasonbeverage jasonbeverage
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: osgearth_package: handling of transparency in TIFF files

Andreas,

I'm suspecting that he has the OSGEARTH_CACHE_PATH environment variable set to setup a global cache.  I tested the data in test.zip with VS 2010 and GDAL 1.9 and it worked as expected.  It generated ~50 tiles and skipped over ones that were transparent.

Jason

On Fri, Apr 27, 2012 at 10:21 AM, Andreas Roth [via osgEarth] <[hidden email]> wrote:
Hi,

in the .earth file posted by rammann no cache is specified, so the osgearth does not cache any tiles.
<?xml version="1.0"?> 
<map version="2"> 
	<image name="test" driver="gdal"> 
		<url>test.tif</url> 
	</image> 
</map> 

@jasonbeverage: Have you tested the osgearth_packager with the provided TIFF-file and .earth file with GDAL 1.9 without any trouble?

Andreas




If you reply to this email, your message will be added to the discussion below:
http://forum.osgearth.org/osgearth-package-handling-of-transparency-in-TIFF-files-tp7483503p7506308.html
To start a new topic under osgEarth, email [hidden email]
To unsubscribe from osgEarth, click here.
NAML

Andreas Roth Andreas Roth
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: osgearth_package: handling of transparency in TIFF files

Hi Jason,

thanks for the clarification. I can assure you that the environment variable was not set for the TMS build process. Have you made any modification to the GDAL sources?
On monday i will check the osgearth_packager with a 3rdparty GDAL 1.9 binary to check if our GDAL is faulty.

Andreas
rammann rammann
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: osgearth_package: handling of transparency in TIFF files

Hi Jason,

did you use the zip from my first or second post? The first file is working fine, only the second one (4 mb) leads to the described issues.
Loading...