PROJ4 >= 6 version not working correctly?

classic Classic list List threaded Threaded
16 messages Options
Nikkitta Nikkitta
Reply | Threaded
Open this post in threaded view
|

PROJ4 >= 6 version not working correctly?

Hi,

I'm using osgearth 2.10.2 with latest dependencies libs built from sources:
- PROJ4  -  6.2.0
- GDAL   -  3.0.1

Everything works almost fine and I even didn't notice this problem for several months!

The thing is, the imagery data is shifted north for about 0.2 degrees.
This behaviour was already described in these posts:
http://forum.osgearth.org/SOLVED-osgEarth-3-0-XYZImageLayer-issue-Tiles-are-shifted-td7592577.html
http://forum.osgearth.org/MVT-rendering-problems-in-version-2-10-1-td7592654.html

But as far as I understood they just downgraded to earlier GDAL. And I think it is quite an issue and it should be solved, because GDAL and PROJ4 started to develop rapidly.

Here is the image depicting described shift, you can see elevation data in the sea:
I use osgearth_viewer with openstreetmap.earth with added readymap elevation


That only happens with driver="xyz". I tried tms driver and gdal to load ECW - both of them works fine.

That happens on Windows 10 x64 MSVC 2019/2015 and on linux (Ubuntu-like).
I will attach the output with OSGEARTH_NOTIFY_LEVEL=INFO:


[osgEarth]  [Map] cache=none; policy=read-write; bin=no
[osgEarth]  [TerrainLayer] Layer "readymap_elevation" Creating "tms" driver
[osgEarth]  [TMSTileSource] TMS tile map datestamp = Thu, 01 Jan 1970 00:00:00 GMT
[osgEarth]  override vdatum = egm96, profile vdatum = 
[osgEarth]  [VerticalDatum] Initializing vertical datum: egm96
[osgEarth]  [TerrainLayer] Layer "readymap_elevation" Override profile: [srs=unknown, min=-180,-90 max=180,90 lod0=2,1 vdatum=EGM96]
[osgEarth]  [TerrainLayer] Layer "readymap_elevation" Profile=[srs=unknown, min=-180,-90 max=180,90 lod0=2,1 vdatum=EGM96]
[osgEarth]  [TerrainLayer] Layer "readymap_elevation" cache=none; policy=read-write; bin=no
[osgEarth]  [TerrainLayer] Layer "osm_mapnik" Creating "xyz" driver
[osgEarth]  [TerrainLayer] Layer "osm_mapnik" Override profile: [srs=unknown, min=-20037508.34278925,-20037508.34278925 max=20037508.34278925,20037508.34278925 lod0=1,1 vdatum=geodetic]
[osgEarth]  [TerrainLayer] Layer "osm_mapnik" Profile=[srs=unknown, min=-20037508.34278925,-20037508.34278925 max=20037508.34278925,20037508.34278925 lod0=1,1 vdatum=geodetic]
[osgEarth]  [TerrainLayer] Layer "osm_mapnik" cache=none; policy=no-cache; bin=no
[osgEarth]  [engine_rex] Activated!
[osgEarth]  [Capabilities] osgEarth Version: 2.10.2 
[osgEarth]  [Capabilities] OSG Version:      3.6.4
[osgEarth]  [Capabilities] GDAL Version:     3.0.1
[osgEarth]  [Capabilities] Detected hardware capabilities:
[osgEarth]  [Capabilities]   Vendor = NVIDIA Corporation
[osgEarth]  [Capabilities]   Renderer = GeForce GTX 580/PCIe/SSE2
[osgEarth]  [Capabilities]   Version = 4.6.0 NVIDIA 391.35
[osgEarth]  [Capabilities]   Core Profile = no
[osgEarth]  [Capabilities]   Max GPU texture units = 32
[osgEarth]  [Capabilities]   Max GPU texture coord indices = 8
[osgEarth]  [Capabilities]   Max GPU attributes = 16
[osgEarth]  [Capabilities]   Depth buffer bits = 24
[osgEarth]  [Capabilities]   Max texture size = 16384
[osgEarth]  [Capabilities]   GLSL = yes
[osgEarth]  [Capabilities]   GLSL Version = 460
[osgEarth]  [Capabilities]   Texture arrays = yes
[osgEarth]  [Capabilities]   draw instanced = yes
[osgEarth]  [Capabilities]   Texture buffers = yes
[osgEarth]  [Capabilities]   Texture buffer max size = 134217728
[osgEarth]  [Capabilities]   Compression = ARB S3 RG
[osgEarth]  [Map] Map profile is: [srs=unknown, min=-180,-90 max=180,90 lod0=2,1 vdatum=geodetic]
[osgEarth]  [PagerLoader] Merges per frame = 20
[osgEarth]  [TerrainResources] Texture unit 0 reserved for Terrain Color
[osgEarth]  [TerrainResources] Texture unit 1 reserved for Terrain Elevation
[osgEarth]  [TerrainResources] Texture unit 2 reserved for Terrain Normals
[osgEarth]  [TerrainResources] Texture unit 3 reserved for Terrain Parent Color


Any suggestions what could it be?
Best regards,
Nikita Petrov
plevy plevy
Reply | Threaded
Open this post in threaded view
|

Re: PROJ4 >= 6 version not working correctly?

Nikita,

We have been tracking with the vcpkg version of gdal which is still at 2.4.1.  Let me get 3.0.1 building again and see if I can reproduce it.

Thank you
--
Paul Levy : Pelican Mapping


On Thu, Sep 5, 2019 at 9:47 AM Nikkitta [via osgEarth] <[hidden email]> wrote:
Hi,

I'm using osgearth 2.10.2 with latest dependencies libs built from sources:
- PROJ4  -  6.2.0
- GDAL   -  3.0.1

Everything works almost fine and I even didn't notice this problem for several months!

The thing is, the imagery data is shifted north for about 0.2 degrees.
This behaviour was already described in these posts:
http://forum.osgearth.org/SOLVED-osgEarth-3-0-XYZImageLayer-issue-Tiles-are-shifted-td7592577.html
http://forum.osgearth.org/MVT-rendering-problems-in-version-2-10-1-td7592654.html

But as far as I understood they just downgraded to earlier GDAL. And I think it is quite an issue and it should be solved, because GDAL and PROJ4 started to develop rapidly.

Here is the image depicting described shift, you can see elevation data in the sea:
I use osgearth_viewer with openstreetmap.earth with added readymap elevation


That only happens with driver="xyz". I tried tms driver and gdal to load ECW - both of them works fine.

That happens on Windows 10 x64 MSVC 2019/2015 and on linux (Ubuntu-like).
I will attach the output with OSGEARTH_NOTIFY_LEVEL=INFO:

[osgEarth]  [Map] cache=none; policy=read-write; bin=no
[osgEarth]  [TerrainLayer] Layer "readymap_elevation" Creating "tms" driver
[osgEarth]  [TMSTileSource] TMS tile map datestamp = Thu, 01 Jan 1970 00:00:00 GMT
[osgEarth]  override vdatum = egm96, profile vdatum = 
[osgEarth]  [VerticalDatum] Initializing vertical datum: egm96
[osgEarth]  [TerrainLayer] Layer "readymap_elevation" Override profile: [srs=unknown, min=-180,-90 max=180,90 lod0=2,1 vdatum=EGM96]
[osgEarth]  [TerrainLayer] Layer "readymap_elevation" Profile=[srs=unknown, min=-180,-90 max=180,90 lod0=2,1 vdatum=EGM96]
[osgEarth]  [TerrainLayer] Layer "readymap_elevation" cache=none; policy=read-write; bin=no
[osgEarth]  [TerrainLayer] Layer "osm_mapnik" Creating "xyz" driver
[osgEarth]  [TerrainLayer] Layer "osm_mapnik" Override profile: [srs=unknown, min=-20037508.34278925,-20037508.34278925 max=20037508.34278925,20037508.34278925 lod0=1,1 vdatum=geodetic]
[osgEarth]  [TerrainLayer] Layer "osm_mapnik" Profile=[srs=unknown, min=-20037508.34278925,-20037508.34278925 max=20037508.34278925,20037508.34278925 lod0=1,1 vdatum=geodetic]
[osgEarth]  [TerrainLayer] Layer "osm_mapnik" cache=none; policy=no-cache; bin=no
[osgEarth]  [engine_rex] Activated!
[osgEarth]  [Capabilities] osgEarth Version: 2.10.2 
[osgEarth]  [Capabilities] OSG Version:      3.6.4
[osgEarth]  [Capabilities] GDAL Version:     3.0.1
[osgEarth]  [Capabilities] Detected hardware capabilities:
[osgEarth]  [Capabilities]   Vendor = NVIDIA Corporation
[osgEarth]  [Capabilities]   Renderer = GeForce GTX 580/PCIe/SSE2
[osgEarth]  [Capabilities]   Version = 4.6.0 NVIDIA 391.35
[osgEarth]  [Capabilities]   Core Profile = no
[osgEarth]  [Capabilities]   Max GPU texture units = 32
[osgEarth]  [Capabilities]   Max GPU texture coord indices = 8
[osgEarth]  [Capabilities]   Max GPU attributes = 16
[osgEarth]  [Capabilities]   Depth buffer bits = 24
[osgEarth]  [Capabilities]   Max texture size = 16384
[osgEarth]  [Capabilities]   GLSL = yes
[osgEarth]  [Capabilities]   GLSL Version = 460
[osgEarth]  [Capabilities]   Texture arrays = yes
[osgEarth]  [Capabilities]   draw instanced = yes
[osgEarth]  [Capabilities]   Texture buffers = yes
[osgEarth]  [Capabilities]   Texture buffer max size = 134217728
[osgEarth]  [Capabilities]   Compression = ARB S3 RG
[osgEarth]  [Map] Map profile is: [srs=unknown, min=-180,-90 max=180,90 lod0=2,1 vdatum=geodetic]
[osgEarth]  [PagerLoader] Merges per frame = 20
[osgEarth]  [TerrainResources] Texture unit 0 reserved for Terrain Color
[osgEarth]  [TerrainResources] Texture unit 1 reserved for Terrain Elevation
[osgEarth]  [TerrainResources] Texture unit 2 reserved for Terrain Normals
[osgEarth]  [TerrainResources] Texture unit 3 reserved for Terrain Parent Color


Any suggestions what could it be?
Best regards,
Nikita Petrov



If you reply to this email, your message will be added to the discussion below:
http://forum.osgearth.org/PROJ4-6-version-not-working-correctly-tp7592785.html
To start a new topic under osgEarth, email [hidden email]
To unsubscribe from osgEarth, click here.
NAML
Nikkitta Nikkitta
Reply | Threaded
Open this post in threaded view
|

Re: PROJ4 >= 6 version not working correctly?

Paul,

Did you manage to reproduce the problem?
Best regards,
Nikita Petrov
plevy plevy
Reply | Threaded
Open this post in threaded view
|

Re: PROJ4 >= 6 version not working correctly?

Building gdal right now actually.  Did you build from source or use Conda binaries?

--
Paul Levy : Pelican Mapping


On Wed, Sep 11, 2019 at 3:14 PM Nikkitta [via osgEarth] <[hidden email]> wrote:
Paul,

Did you manage to reproduce the problem?
Best regards,
Nikita Petrov



If you reply to this email, your message will be added to the discussion below:
http://forum.osgearth.org/PROJ4-6-version-not-working-correctly-tp7592785p7592796.html
To start a new topic under osgEarth, email [hidden email]
To unsubscribe from osgEarth, click here.
NAML
Nikkitta Nikkitta
Reply | Threaded
Open this post in threaded view
|

Re: PROJ4 >= 6 version not working correctly?

I build PROJ, GDAL and all other dependent libs from source with CMake, NMake, MSVC2015/2019.
Best regards,
Nikita Petrov
plevy plevy
Reply | Threaded
Open this post in threaded view
|

Re: PROJ4 >= 6 version not working correctly?

OK perfect,  While I was building from source, I noticed there are at least a couple binary packages, Conda and Gisinternals.  I got it built yesterday and will see about reproducing your issue.

Thank you
--
Paul Levy : Pelican Mapping


On Thu, Sep 12, 2019 at 2:22 AM Nikkitta [via osgEarth] <[hidden email]> wrote:
I build PROJ, GDAL and all other dependent libs from source with CMake, NMake, MSVC2015/2019.
Best regards,
Nikita Petrov



If you reply to this email, your message will be added to the discussion below:
http://forum.osgearth.org/PROJ4-6-version-not-working-correctly-tp7592785p7592798.html
To start a new topic under osgEarth, email [hidden email]
To unsubscribe from osgEarth, click here.
NAML
plevy plevy
Reply | Threaded
Open this post in threaded view
|

Re: PROJ4 >= 6 version not working correctly?

Nikita, it looks normal to me at this viewpoint, can you double check at this point?
<viewpoint>
  <heading>39.7017</heading>
  <pitch>-11.1493</pitch>
  <range>69248.3m</range>
  <long>-155.3262116987146</long>
  <lat>19.25977122570158</lat>
  <height>102.1435620160773</height>
  <srs>+proj=longlat +datum=WGS84 +no_defs</srs>
</viewpoint>

I am on a later version of OSG:
[osgEarth]  [Capabilities] osgEarth Version: 2.10.2
[osgEarth]  [Capabilities] OSG Version:      3.7.0
[osgEarth]  [Capabilities] GDAL Version:     3.0.1

.earth

<!--
OpenStreetMap's Mapnik "Slippy Map" - Geocentric.
See:
http://wiki.openstreetmap.org/wiki/Slippy_map_tilenames
-->

<map name="OpenStreetMap" type="geocentric" version="2">
    <image name="osm_mapnik" driver="xyz">
        <url>http://[abc].<a href="http://tile.openstreetmap.org/{z}/{x}/{y}.png">tile.openstreetmap.org/{z}/{x}/{y}.png</url>
        <profile>spherical-mercator</profile>
        <cache_policy usage="none"/>
        <attribution>&#169;OpenStreetMap contributors</attribution>
    </image>

    <elevation name="readymap_elevation" driver="tms" vdatum="egm96">
        <url>http://readymap.org/readymap/tiles/1.0.0/116/</url>
    </elevation>

    <options>
        <lighting>false</lighting>
        <terrain>
            <min_tile_range_factor>8</min_tile_range_factor>
        </terrain>
    </options>

    <xi:include href="viewpoints.xml"/>

</map>




image.png
--
Paul Levy : Pelican Mapping


On Thu, Sep 12, 2019 at 9:50 AM plevy [via osgEarth] <[hidden email]> wrote:
OK perfect,  While I was building from source, I noticed there are at least a couple binary packages, Conda and Gisinternals.  I got it built yesterday and will see about reproducing your issue.

Thank you
--
Paul Levy : Pelican Mapping


On Thu, Sep 12, 2019 at 2:22 AM Nikkitta [via osgEarth] <[hidden email]> wrote:
I build PROJ, GDAL and all other dependent libs from source with CMake, NMake, MSVC2015/2019.
Best regards,
Nikita Petrov



If you reply to this email, your message will be added to the discussion below:
http://forum.osgearth.org/PROJ4-6-version-not-working-correctly-tp7592785p7592798.html
To start a new topic under osgEarth, email [hidden email]
To unsubscribe from osgEarth, click here.
NAML



If you reply to this email, your message will be added to the discussion below:
http://forum.osgearth.org/PROJ4-6-version-not-working-correctly-tp7592785p7592799.html
To start a new topic under osgEarth, email [hidden email]
To unsubscribe from osgEarth, click here.
NAML
Nikkitta Nikkitta
Reply | Threaded
Open this post in threaded view
|

Re: PROJ4 >= 6 version not working correctly?

That's sad, I still have the same shift:


So it looks like I am building dependent libs wrong, but still there are 2 more mentions of this problem in the links of the first post, so I am not the only one who see it.

I am also using stable OSG 3.6.4, but it is doubtful that there could be such changes that will shift image layer in osgearth.

I have my own script that builds all dependent libs from sources - https://github.com/NikkittaP/osg_pack
In sources/versions.txt you can find which lib's versions I am using.
Perhaps if you look at it, this will shed light on the error. Maybe I'm using some known bad versions.

Thank you
Best regards,
Nikita Petrov
plevy plevy
Reply | Threaded
Open this post in threaded view
|

Re: PROJ4 >= 6 version not working correctly?

I just started rebuilding with gisinternals version of gdal3,0 to see if it shows the problem.

I think your github script is private as I get a 404.
 
--
Paul Levy : Pelican Mapping

On Thu, Sep 12, 2019 at 4:49 PM Nikkitta [via osgEarth] <[hidden email]> wrote:
That's sad, I still have the same shift:


So it looks like I am building dependent libs wrong, but still there are 2 more mentions of this problem in the links of the first post, so I am not the only one who see it.

I am also using stable OSG 3.6.4, but it is doubtful that there could be such changes that will shift image layer in osgearth.

I have my own script that builds all dependent libs from sources - https://github.com/NikkittaP/osg_pack
In sources/versions.txt you can find which lib's versions I am using.
Perhaps if you look at it, this will shed light on the error. Maybe I'm using some known bad versions.

Thank you
Best regards,
Nikita Petrov



If you reply to this email, your message will be added to the discussion below:
http://forum.osgearth.org/PROJ4-6-version-not-working-correctly-tp7592785p7592801.html
To start a new topic under osgEarth, email [hidden email]
To unsubscribe from osgEarth, click here.
NAML
Nikkitta Nikkitta
Reply | Threaded
Open this post in threaded view
|

Re: PROJ4 >= 6 version not working correctly?

Indeed it was. Switched it to public.
Best regards,
Nikita Petrov
plevy plevy
Reply | Threaded
Open this post in threaded view
|

Re: PROJ4 >= 6 version not working correctly?

Looking over the previous threads, the first appeared to be a build issue.  He switched to pre-built libraries and it started working.
I am not using pre-built, but using vcpkg to build all the dependencies and then swapping out gdal for the one I built.  

Have you double checked the gdal-data is not finding the proj4 version first?  Not caching old data?


--
Paul Levy : Pelican Mapping


On Thu, Sep 12, 2019 at 5:16 PM Nikkitta [via osgEarth] <[hidden email]> wrote:
Indeed it was. Switched it to public.
Best regards,
Nikita Petrov



If you reply to this email, your message will be added to the discussion below:
http://forum.osgearth.org/PROJ4-6-version-not-working-correctly-tp7592785p7592803.html
To start a new topic under osgEarth, email [hidden email]
To unsubscribe from osgEarth, click here.
NAML
Nikkitta Nikkitta
Reply | Threaded
Open this post in threaded view
|

Re: PROJ4 >= 6 version not working correctly?

What do you mean by "the gdal-data is not finding the proj4 version first"?

I set GDAL_DATA and PROJ_LIB (with proj.db file) variables within my .cpp:

#ifdef _WIN32
    _putenv_s("GDAL_DATA", (app.applicationDirPath() + "/gdal_data/").toStdString().c_str());
    _putenv_s("PROJ_LIB", (app.applicationDirPath() + "/proj_data/").toStdString().c_str());
#endif // _WIN32

Is it wrong? I see no errors in the output except strange srs="unknown" here:
[osgEarth]  [Map] Map profile is: [srs=unknown, min=-180,-90 max=180,90 lod0=2,1 vdatum=geodetic]
Best regards,
Nikita Petrov
plevy plevy
Reply | Threaded
Open this post in threaded view
|

Re: PROJ4 >= 6 version not working correctly?

Nikita,

Is your osm_mapnik layer still showing srs unknown?  Here is mine:

[osgEarth]  [TerrainLayer] Layer "osm_mapnik" Creating "xyz" driver
[osgEarth]  [TerrainLayer] Layer "osm_mapnik" Override profile: [srs=WGS 84 / Pseudo-Mercator, min=-20037508.34278925,-20037508.34278925 max=20037508.34278925,20037508.34278925 lod0=1,1 vdatum=geodetic]
[osgEarth]  [TerrainLayer] Layer "osm_mapnik" Profile=[srs=WGS 84 / Pseudo-Mercator, min=-20037508.34278925,-20037508.34278925 max=20037508.34278925,20037508.34278925 lod0=1,1 vdatum=geodetic]
[osgEarth]  [TerrainLayer] Layer "osm_mapnik" cache=none; policy=no-cache; bin=no

--
Paul Levy : Pelican Mapping


On Fri, Sep 13, 2019 at 5:01 AM Nikkitta [via osgEarth] <[hidden email]> wrote:
What do you mean by "the gdal-data is not finding the proj4 version first"?

I set GDAL_DATA and PROJ_LIB (with proj.db file) variables within my .cpp:

#ifdef _WIN32
    _putenv_s("GDAL_DATA", (app.applicationDirPath() + "/gdal_data/").toStdString().c_str());
    _putenv_s("PROJ_LIB", (app.applicationDirPath() + "/proj_data/").toStdString().c_str());
#endif // _WIN32

Is it wrong? I see no errors in the output except strange srs="unknown" here:
[osgEarth]  [Map] Map profile is: [srs=unknown, min=-180,-90 max=180,90 lod0=2,1 vdatum=geodetic]
Best regards,
Nikita Petrov



If you reply to this email, your message will be added to the discussion below:
http://forum.osgearth.org/PROJ4-6-version-not-working-correctly-tp7592785p7592806.html
To start a new topic under osgEarth, email [hidden email]
To unsubscribe from osgEarth, click here.
NAML
Nikkitta Nikkitta
Reply | Threaded
Open this post in threaded view
|

Re: PROJ4 >= 6 version not working correctly?

Paul,

I apologize for my long absence.

I rebuilt everything with my script and strangely now it works correctly. Tested on Windows (VS2015/VS2019) and on Linux (Debian-like).
The only possible thing that changed since the error existed is an updated version of PROJ - 6.1.1->6.2.0.

My colleagues will also test these updated dependencies libs and if there are any problems I will report back.


Regarding your question about srs - osm_mapnik now shows WGS 84, the same output as yours.
But for the map I still have unknown:

[osgEarth]  [Map] Map profile is: [srs=unknown, min=-180,-90 max=180,90 lod0=2,1 vdatum=geodetic]


Anyway, for now everything works correctly with stable GDAL 3.0.1 and PROJ 6.2.0.
Best regards,
Nikita Petrov
plevy plevy
Reply | Threaded
Open this post in threaded view
|

Re: PROJ4 >= 6 version not working correctly?

Nikita,

That is great news and this thread will be helpful as users begin migrating to gdal 3.0.

Thank you
--
Paul Levy : Pelican Mapping


On Tue, Sep 17, 2019 at 9:32 AM Nikkitta [via osgEarth] <[hidden email]> wrote:
Paul,

I apologize for my long absence.

I rebuilt everything with my script and strangely now it works correctly. Tested on Windows (VS2015/VS2019) and on Linux (Debian-like).
The only possible thing that changed since the error existed is an updated version of PROJ - 6.1.1->6.2.0.

My colleagues will also test these updated dependencies libs and if there are any problems I will report back.


Regarding your question about srs - osm_mapnik now shows WGS 84, the same output as yours.
But for the map I still have unknown:

[osgEarth]  [Map] Map profile is: [srs=unknown, min=-180,-90 max=180,90 lod0=2,1 vdatum=geodetic]


Anyway, for now everything works correctly with stable GDAL 3.0.1 and PROJ 6.2.0.
Best regards,
Nikita Petrov



If you reply to this email, your message will be added to the discussion below:
http://forum.osgearth.org/PROJ4-6-version-not-working-correctly-tp7592785p7592817.html
To start a new topic under osgEarth, email [hidden email]
To unsubscribe from osgEarth, click here.
NAML
remoe remoe
Reply | Threaded
Open this post in threaded view
|

Re: PROJ4 >= 6 version not working correctly?

In reply to this post by Nikkitta
Nikkitta, thanks to dive into this ;)
Remo Eichenberger, Switzerland