osgearth_seed issue

classic Classic list List threaded Threaded
20 messages Options
Stan Stan
Reply | Threaded
Open this post in threaded view
|

osgearth_seed issue

Hi everybody,
I would like to use osgearth_seed to create tms cache for my map data. The tricky bit is, that all I am getting is nice folder hierarchy with files 633B long (jpg format). The osgearth_seed options are as those in official documentation (osgearth_seed --max-level 7 myfile.earth). I expected files with different size/quality in different cache levels/folders. What am I doing anything wrong?

Stan
gwaldron gwaldron
Reply | Threaded
Open this post in threaded view
|

Re: osgearth_seed issue

Stan,
Can you post your .earth file? Does it display correctly in osgviewer? Can you attach one of the jpg's?

Glenn
Glenn Waldron / Pelican Mapping
Stan Stan
Reply | Threaded
Open this post in threaded view
|

Re: osgearth_seed issue

Glenn,
my earth file follows:

<map type="geocentric">
    <cache type="tms">
        <path>G:/mak_cache</path>
        <format>jpg</format>
    </cache>

   <profile>global-geodetic</profile>

   
   <image name="vtul_cze_rasters" driver="gdal">
       <url>G:/sim_maps/Maps</url>
   </image>
</map>

One of the jpg's (level 0) can be downloaded  here:
http://myfreefilehosting.com/f/2b779754c6_0.1MB

I guess it's just a header with no data.

Stan
Stan Stan
Reply | Threaded
Open this post in threaded view
|

Re: osgearth_seed issue

In reply to this post by gwaldron
and yes, in osgviewer everything displays correctly. Once I run it, the cache files begin to grow as I would epected, different sizes in different levels. But I would be glad if osgearth_seed does the work for me :-) and cache "everything".

Stan
jasonbeverage jasonbeverage
Reply | Threaded
Open this post in threaded view
|

Re: osgearth_seed issue

Hi Stan,

What version of osgEarth are you using?  Are the files you are using the "vtul_cze_rasters" folder 3 band or 4 band images, what projection are they in and do they cover the whole earth or are they insets?

Thanks,

Jason

On Fri, Oct 16, 2009 at 9:10 AM, Stan (via Nabble) - No Reply <[hidden email]> wrote:
and yes, in osgviewer everything displays correctly. Once I run it, the cache files begin to grow as I would epected, different sizes in different levels. But I would be glad if osgearth_seed does the work for me :-) and cache "everything".

Stan


View message @ http://n2.nabble.com/osgearth-seed-issue-tp3835290p3835469.html
To start a new topic under osgEarth, email [hidden email]
To unsubscribe from osgEarth, click here.


Stan Stan
Reply | Threaded
Open this post in threaded view
|

Re: osgearth_seed issue

- osgearth_1.2RC2_T2009-08-07.r555 (debug)
- OpenSceneGraph-2.8.2
- gdal16-1.6.2
- curl-7.15.1
- expat-2.0.1
- files projection: GEO(WGS84)
- files cover only The Czech Republic area
- you can download one of them here:
http://myfreefilehosting.com/f/f6f246d07a_13.1MB

there is nothig to be seen in it (it's one of the edge files), but it is the smallest one i have.

Stan
jasonbeverage jasonbeverage
Reply | Threaded
Open this post in threaded view
|

Re: osgearth_seed issue

Hi Stan,

Thanks for sending the sample file, it helped to see what was going on.

When you run osgearth_seed with no --bounds argument it tries to cache the whole map extents (-180,-90,180,90).  In your case, you don't have any data for most of the world, just around the Czech Republic so the GDAL driver returns a 1x1 transparent pixel for tiles that do not overlap the data, which are the 633 byte files you have.  For the file you gave me I was able to cache just the area around the file using:

osgearth_seed  --bounds 16.75 50.50 17.0 50.75 --max-level 10 mymap.earth

You can see one actual data tile at 10\1120\799.jpg for instance.

Also, I would recommend using PNG as a cache format rather than JPEG when you are dealing with insets so that tiles that partially overlap the data will have transparency where there is no data rather than black.

Thanks!

Jason



On Fri, Oct 16, 2009 at 9:49 AM, Stan (via Nabble) - No Reply <[hidden email]> wrote:
- osgearth_1.2RC2_T2009-08-07.r555 (debug)
- OpenSceneGraph-2.8.2
- gdal16-1.6.2
- curl-7.15.1
- expat-2.0.1
- files projection: GEO(WGS84)
- files cover only The Czech Republic area
- you can download one of them here:
http://myfreefilehosting.com/f/f6f246d07a_13.1MB

there is nothig to be seen in it (it's one of the edge files), but it is the smallest one i have.

Stan



View message @ http://n2.nabble.com/osgearth-seed-issue-tp3835290p3835679.html
To start a new topic under osgEarth, email [hidden email]
To unsubscribe from osgEarth, click here.


Stan Stan
Reply | Threaded
Open this post in threaded view
|

Re: osgearth_seed issue

Thank you very much, it is the way forward. I'll find out the bounds of my maps and give it a try. Regarding PNG, it seems that i don't have the correct plugin, because osgearth_seed is giving me the warning: Could not find plugin to write image to file "filename". Could you give me any advice were i can get that plugin?

Stan
jasonbeverage jasonbeverage
Reply | Threaded
Open this post in threaded view
|

Re: osgearth_seed issue

Hi Stan,

The PNG plugin should be built when you build OpenSceneGraph.  You have to have libpng available which should come with the 3rdParty dependencies available on the OSG website.

Check to see if osgdb_png.dll exists with your other OSG plugins.

Thanks,

Jason

On Fri, Oct 16, 2009 at 10:36 AM, Stan (via Nabble) - No Reply <[hidden email]> wrote:
Thank you very much, it is the way forward. I'll find out the bounds of my maps and give it a try. Regarding PNG, it seems that i don't have the correct plugin, because osgearth_seed is giving me the warning: Could not find plugin to write image to file "filename". Could you give me any advice were i can get that plugin?

Stan


View message @ http://n2.nabble.com/osgearth-seed-issue-tp3835290p3835887.html
To start a new topic under osgEarth, email [hidden email]
To unsubscribe from osgEarth, click here.


Stan Stan
Reply | Threaded
Open this post in threaded view
|

Re: osgearth_seed issue

Hi Jason,
i have the osgdb_png.dll plugin and i am using libpng13.dll because it is specificaly required by some other dll. Could that be an issue?

Stan
jasonbeverage jasonbeverage
Reply | Threaded
Open this post in threaded view
|

Re: osgearth_seed issue

As long as the osgdb_png.dll was built against libpng13.lib  you should be ok.  Also make sure zlib1.dll is in your path as it is another dependency of the png plugin.

Jason

On Fri, Oct 16, 2009 at 11:02 AM, Stan (via Nabble) - No Reply <[hidden email]> wrote:
Hi Jason,
i have the osgdb_png.dll plugin and i am using libpng13.dll because it is specificaly required by some other dll. Could that be an issue?

Stan


View message @ http://n2.nabble.com/osgearth-seed-issue-tp3835290p3836018.html
To start a new topic under osgEarth, email [hidden email]
To unsubscribe from osgEarth, click here.


Stan Stan
Reply | Threaded
Open this post in threaded view
|

Re: osgearth_seed issue

Hi Jason,
thank you for your help. I'll check all plugin dependencies again. I'll will post my results.

Stan

Stan Stan
Reply | Threaded
Open this post in threaded view
|

Re: osgearth_seed issue

In reply to this post by jasonbeverage
Hi Jason,
I think i am still missing the osgearth_seed phylosophy. My idea was that if I have osgearth_seed cached my map sources with max. level i'll eventually get data much bigger than the source data because it would contain both low and high resolution files up to the resolution of a source files. Cache like that I would be able to use as a real data source with no further need of the original data as referenced in documentation.  

  <image name="bluemarble" driver="tms">
        <url>c:/osgearth_cache/bluemarble/tms.xml</url>
  </image>

Even with the correct bounds I am still getting cache with data much smaller then the original data. E.g. source_file.tif [500MB], cache_dir[20MB].

Can I tell osgearth_seed that I need all resolutions? Up to source file resolution? Is there any other documentation then this?
http://wush.net/trac/osgearth/wiki/EarthFileCaching


Thanks,
Stan
jasonbeverage jasonbeverage
Reply | Threaded
Open this post in threaded view
|

Re: osgearth_seed issue

Hi Stan,

Are you passing in a max level to osgearth_seed?  If you don't then it assumes a max level of 5 which probably isn't enough to get the highest resolution for your source data.  Try bumping it up to level 14 or so and see what happens.

Thanks,

Jason

On Fri, Oct 16, 2009 at 1:39 PM, Stan (via Nabble) - No Reply <[hidden email]> wrote:
Hi Jason,
I think i am still missing the osgearth_seed phylosophy. My idea was that if I have osgearth_seed cached my map sources with max. level i'll eventually get data much bigger than the source data because it would contain both low and high resolution files up to the resolution of a source files. Cache like that I would be able to use as a real data source with no further need of the original data as referenced in documentation.  

  <image name="bluemarble" driver="tms">
        <url>c:/osgearth_cache/bluemarble/tms.xml</url>
  </image>

Even with the correct bounds I am still getting cache with data much smaller then the original data. E.g. source_file.tif [500MB], cache_dir[20MB].

Can I tell osgearth_seed that I need all resolutions? Up to source file resolution? Is there any other documentation then this?
http://wush.net/trac/osgearth/wiki/EarthFileCaching


Thanks,
Stan



View message @ http://n2.nabble.com/osgearth-seed-issue-tp3835290p3837068.html
To start a new topic under osgEarth, email [hidden email]
To unsubscribe from osgEarth, click here.


Stan Stan
Reply | Threaded
Open this post in threaded view
|

Re: osgearth_seed issue

Hi Jason,
i have tried different levels 10, 12, 15 with more or less the same results. Is it correct to find out the bounding rectangle of the data I have and pass it to osgearth_seed? There are no gaps among source files - the whole country is covered and the source data ends with the border. I mostly get nice tiles in levels 0-2 (the whole data/country overview) and black tiles in higher levels.

Thanks
Stan
jasonbeverage jasonbeverage
Reply | Threaded
Open this post in threaded view
|

Re: osgearth_seed issue

Hi Stan,

Can you post the command line you are using?  You'll sometimes see this if you swap your lat and lons when specifying the bounds.  The correct order is min_lon min_lat max_lon max_lat

Jason

On Fri, Oct 16, 2009 at 4:31 PM, Stan (via Nabble) - No Reply <[hidden email]> wrote:
Hi Jason,
i have tried different levels 10, 12, 15 with more or less the same results. Is it correct to find out the bounding rectangle of the data I have and pass it to osgearth_seed? There are no gaps among source files - the whole country is covered and the source data ends with the border. I mostly get nice tiles in levels 0-2 (the whole data/country overview) and black tiles in higher levels.

Thanks
Stan


View message @ http://n2.nabble.com/osgearth-seed-issue-tp3835290p3838048.html
To start a new topic under osgEarth, email [hidden email]
To unsubscribe from osgEarth, click here.


Stan Stan
Reply | Threaded
Open this post in threaded view
|

Re: osgearth_seed issue

Hi Jason,
you were right, i am so used to "TopLeft BottomRight" schema that i swapped lats. Now it works nice till level 11. When I run osgearth_seed with level 12 or higher, it crashes after some time. When I run it again, it crashes somewhere else seeding a different tile. I attach call stack if it helps:
http://myfreefilehosting.com/f/1df84b1c2e_0.1MB (It seems that the mem allocator is passed incorrect FileName pointer.)

My second issue is with seeding heightfields. I have DTED2 data and when seeding osgearth_seed outputs warning: [osgEarth::ImageUtils::convertToRGB] pixelFormat 6409 not yet supported. Do I have to wait for an upgrade or is there any other possibility?

Thanks,
Stan


       
jasonbeverage jasonbeverage
Reply | Threaded
Open this post in threaded view
|

Re: osgearth_seed issue

Hi Stan,

There is a memory leak in the cache seeding with osgEarth 1.2 that has been fixed in later revisions.  I've attached a path that should fix 1.2 until we do a 1.3 release.

As for caching heightfields, you have to be careful about the cache format  you are using since heightfield data is single band floating point data.  Most OSG formats do not handle it correctly.  DDS is the recommended format if you are using OSG 2.6.X and Tiff is the recommended format if you are using OSG 2.8.X or better.  What you need to do is override the cache format for heightfields and tell them not to write to JPG by adding a cache block like:

<map>
  <cache>
    <path>cache</path>
    <format>jpg</format>
  </cache>

  <image>
    ...
  </image>

  <heightfield>
    <!--Override the cache format here-->
    <cache>
       <format>tiff</format>
    </cache>
  </heightfield>

</map>

Thanks,

Jason

On Sat, Oct 17, 2009 at 8:45 AM, Stan (via Nabble) - No Reply <[hidden email]> wrote:
Hi Jason,
you were right, i am so used to "TopLeft BottomRight" schema that i swapped lats. Now it works nice till level 11. When I run osgearth_seed with level 12 or higher, it crashes after some time. When I run it again, it crashes somewhere else seeding a different tile. I attach call stack if it helps:
http://myfreefilehosting.com/f/1df84b1c2e_0.1MB (It seems that the mem allocator is passed incorrect FileName pointer.)

My second issue is with seeding heightfields. I have DTED2 data and when seeding osgearth_seed outputs warning: [osgEarth::ImageUtils::convertToRGB] pixelFormat 6409 not yet supported. Do I have to wait for an upgrade or is there any other possibility?

Thanks,
Stan


       


View message @ http://n2.nabble.com/osgearth-seed-issue-tp3835290p3840633.html
To start a new topic under osgEarth, email [hidden email]
To unsubscribe from osgEarth, click here.



CacheSeedMemLeakFix.patch (852 bytes) Download Attachment
Stan Stan
Reply | Threaded
Open this post in threaded view
|

Re: osgearth_seed issue

Hi Jason,
thank you for the patch. I've thought that my heightfields caching issue could be related to the cache file format so I did:

<map>
  <cache>
    <path>cache_path</path>
    <format>tiff</format>
  </cache>
  <image>
   ...
  </image>
  <heightfield>
  ...
  </heightfield>
</map>

Is there a difference between global cache settings and specific override? Nevertheless, I'll be specific and override heightfields cache format and see what happen. I'll post my results tmrw...

Thanks
Stan
Stan Stan
Reply | Threaded
Open this post in threaded view
|

Re: osgearth_seed issue

In reply to this post by jasonbeverage
Hi Jason,
osgearth_seed with the patch applied works great and the heightfield caching into tiff as well. All issues resolved. Thank you for your help!

Stan