[solved] mkgmap: double processing of ways due to oneway:bicycle=no

argetlam · 2022-06-19 18:52:21

Toubleshooting unwanted double type definitions (same way getting two type assignments on level 0 although both defined without "continue") I was able to pin down some seemingly strange behaviour. Mkgmap seems to process ways (the lines file) twice if they are tagged with oneway:bicycle=no, and unexpected tag changes are made for the second run.

If oneway:bicyle=no ist tagged for the original OSM way:
- oneway=yes is set to oneway=no for the second run
- access=no is set for the second run
hence different conditions from the lines file may be met and different types may be assigned during either run (-> double assignment)

Since I couldn't find that stemming from my code or the standard includes, I've peppered my lines file with debug echos.

Mkgmap version 4896
Time started: Sun Jun 19 19:29:06 CEST 2022
Way 23681312 lines2 checked_element_name=Lippmauer
Way 23681312 lines3 access=null oneway=yes oneway:bicycle=no
Way 23681312 lines549 check on access
Way 23681312 lines562 no match, access=null oneway=yes oneway:bicycle=no
Way 23681312 lines680 l1 check on living street w/ poor surface
Way 23681312 lines685 match->def l1 w/ continue, access=null oneway=yes oneway:bicycle=no
Way 23681312 lines898 1st line <finalize> section access=null oneway=yes oneway:bicycle=no
Way 23681312 lines904 last line <finalize> section access=null oneway=yes oneway:bicycle=no
Way 23681312 lines699 l0 check on living street w/ poor surface
Way 23681312 lines704 match->def l0 w/o continue, access=null oneway=yes oneway:bicycle=no
Way 23681312 lines898 1st line <finalize> section access=null oneway=yes oneway:bicycle=no
Way 23681312 lines904 last line <finalize> section access=null oneway=yes oneway:bicycle=no
Way 23681312 lines2 checked_element_name=Lippmauer
Way 23681312 lines3 access=no oneway=no oneway:bicycle=no
Way 23681312 lines549 check on access
Way 23681312 lines557 match->def l1 w/o continue, access=no oneway=no oneway:bicycle=no
Way 23681312 lines898 1st line <finalize> section access=no oneway=no oneway:bicycle=no
Way 23681312 lines904 last line <finalize> section access=no oneway=no oneway:bicycle=no
Number of MapFailedExceptions: 0
Number of ExitExceptions: 0
Time finished: Sun Jun 19 19:30:21 CEST 2022
Total time taken: 1 minute 15 seconds

The behaviour does not change, if oneway and oneway:bicycle tags are deleted in the very first lines of the lines file, hence seem not to be caused_by or controllable by the style? Uncommented lines 5 to 8 (see below) result in:

Mkgmap version 4896
Time started: Sun Jun 19 19:33:48 CEST 2022
Way 23681312 lines2 checked_element_name=Lippmauer
Way 23681312 lines3 access=null oneway=yes oneway:bicycle=no
Way 23681312 lines8 access=null oneway=null oneway:bicycle=null
Way 23681312 lines549 check on access
Way 23681312 lines562 no match, access=null oneway=null oneway:bicycle=null
Way 23681312 lines680 l1 check on living street w/ poor surface
Way 23681312 lines685 match->def l1 w/ continue, access=null oneway=null oneway:bicycle=null
Way 23681312 lines898 1st line <finalize> section access=null oneway=null oneway:bicycle=null
Way 23681312 lines904 last line <finalize> section access=null oneway=null oneway:bicycle=null
Way 23681312 lines699 l0 check on living street w/ poor surface
Way 23681312 lines704 match->def l0 w/o continue, access=null oneway=null oneway:bicycle=null
Way 23681312 lines898 1st line <finalize> section access=null oneway=null oneway:bicycle=null
Way 23681312 lines904 last line <finalize> section access=null oneway=null oneway:bicycle=null
Way 23681312 lines2 checked_element_name=Lippmauer
Way 23681312 lines3 access=no oneway=no oneway:bicycle=no
Way 23681312 lines8 access=no oneway=no oneway:bicycle=null
Way 23681312 lines549 check on access
Way 23681312 lines557 match->def l1 w/o continue, access=no oneway=no oneway:bicycle=null
Way 23681312 lines898 1st line <finalize> section access=no oneway=no oneway:bicycle=null
Way 23681312 lines904 last line <finalize> section access=no oneway=no oneway:bicycle=null
Number of MapFailedExceptions: 0
Number of ExitExceptions: 0
Time finished: Sun Jun 19 19:35:03 CEST 2022
Total time taken: 1 minute 14 seconds

My takes:
- during the first run, the monitored tags seem to stay unchanged
- the second run seems to re-read the original elements tags (deleted oneway:bicycle in second sample) but operate on modified oneway and access tags

My questions:
- why does mkgmap run twice on these elements?
- what causes the access and oneway tags to be re-set for the second run?
- even if this was intended behaviour, shouldn't only one run actually define types?
- or: how to avoid that double definition safely?

The relevant lines from my lines-file (linenumbers added for reference, original statements commented: 544 ff., 676 ff., 695 ff.):

1 highway=* {set NameCheck='Lippmauer'}
2 name=${NameCheck} {echo "lines2 checked_element_name=${name}"}
3 name=${NameCheck} {echo "lines3 access=${access} oneway=${oneway} oneway:bicycle=${oneway:bicycle}"}

5 # oneway=yes  {delete oneway}
6 # oneway:bicycle=no  {delete oneway:bicycle}

8 # name=${NameCheck} {echo "lines8 access=${access} oneway=${oneway} oneway:bicycle=${oneway:bicycle}"}

# Line 1 to check one way at a time by name (since I don't know how to match by element number).
# Lines 5 and 6 to prove, that its the original OSM elments tag that matter here, not anything from the style.

541 # Private, no Access
542 # ~~~~~~~~~~~~~~~~~~

544 # (highway='track' | highway='raceway' | highway='unclassified' |  highway='minor' | highway='service' 
545 #     | highway='residential' |  highway='road' |  highway='grade1_ruts' | highway='living_street')
546 #   & (access='private' | access='no')                     [0x10e04  level 0]

# don't argue the unsophisticated use of "access=no", needs more like  "& (bicycles != 'yes' | bicycles != 'permissive')"
  
549 name=${NameCheck} {echo "lines549 check on access"}
                                          
551 if (highway='track' | highway='raceway' | highway='unclassified' |  highway='minor' 
552     | highway='service'  | highway='residential' |  highway='road' |  highway='grade1_ruts' 
553     | highway='living_street') then
554     if (access='private' | access='no') then
555     # replaces "&" not parsed in "if" line
556        if (name=${NameCheck}) then
557           ()                  {echo "lines557 match->def l1 w/o continue, access=${access} oneway=${oneway} oneway:bicycle=${oneway:bicycle}"} 
558        end
559           ()                                                              [0x10e04  level 0]
560     else
561        if (name=${NameCheck}) then
562           ()                                    {echo "lines562 no match, access=${access} oneway=${oneway} oneway:bicycle=${oneway:bicycle}"}  
563        end        
564    end
565 end


676 # highway='living_street'
677 #  & (surface=cobblestone | surface=paving_stones | surface=sett | surface=fine_gravel | surface=shells)
678 #                                                                        [0x1A    road_class=1 road_speed=1 level 1-1 continue]

680 name=${NameCheck} {echo "lines680 l1 check on living street w/ poor surface"}
                                          
682 if (highway='living_street'
683    & (surface=cobblestone | surface=paving_stones | surface=sett | surface=fine_gravel | surface=shells)) then
684       if (name=${NameCheck}) then
685          ()                  {echo "lines685 match->def l1 w/ continue, access=${access} oneway=${oneway} oneway:bicycle=${oneway:bicycle}"} 
686       end
687       ()                                   { name'${KurzBez}'}         [0x1A    road_class=1 road_speed=1 level 1-1 continue]
688 end


695 # highway='living_street'
696 #  & (surface=cobblestone | surface=paving_stones | surface=sett | surface=fine_gravel | surface=shells)
697 #                                            { name'${KurzBez}'}         [0x1A    road_class=1 road_speed=1 level 0]

699 name=${NameCheck} {echo "lines699 l0 check on living street w/ poor surface"}
                                          
701 if (highway='living_street'
702    & (surface=cobblestone | surface=paving_stones | surface=sett | surface=fine_gravel | surface=shells)) then
703       if (name=${NameCheck}) then
704          ()                  {echo "lines704 match->def l0 w/o continue, access=${access} oneway=${oneway} oneway:bicycle=${oneway:bicycle}"} 
705       end
706       ()                                   { name'${KurzBez}'}         [0x1A    road_class=1 road_speed=1 level 0]
707 end

896 <finalize>
897 # The finalizer section is executed for each element when a rule with an element type matches
898 name=${NameCheck} {echo "lines898 1st line <finalize> section access=${access} oneway=${oneway} oneway:bicycle=${oneway:bicycle}"}
899 include 'inc/address';
900 # calculate the road speed based on maxspeed tag
901 include 'inc/roadspeed'; 
902 # calculate the access rules
903 include 'inc/access';
904 name=${NameCheck} {echo "lines904 last line <finalize> section access=${access} oneway=${oneway} oneway:bicycle=${oneway:bicycle}"}

//Felix

Last edited by argetlam (2022-07-03 16:27:44)

GerdP · 2022-06-19 18:54:48

Do you use option --make-opposite-cycleways?

argetlam · 2022-06-19 18:59:23

yes

GerdP · 2022-06-19 19:02:18

What you see is the effect of that option.

argetlam · 2022-06-19 19:14:25

Without that, the double processing is gone. (Quick shot)

Did cost me quite some time, but taught me some debugging at least :-)

So its the newly created way for the both-direction cycleway that is processed with the same way number in the second run?
Couldn't that way then be tagged highway=cycleway and not (seemingly) inherit the highway tag from the original? That should have avoided the access mismatch in the second run in my case at least...

Is there a way to directly match elements by their number?

Thanks

//Felix

GerdP · 2022-06-19 19:30:24

The additional way gets the tag mkgmap:synthesised=yes , so you can decide what you do with it in the style.
You can try to use style function osmid(). I must confess that I am surprised that the copy has the same id, normally mkgmap uses an internal id for copies or generated ways. Seems this was changed in 2014, don't remember the reason.
Edit: I prefer to use echotags so that I see all tags of an object

Last edited by GerdP (2022-06-19 19:35:01)

argetlam · 2022-06-19 21:02:42

Good hint.

But although the manual says

The echotags action prints the element id, all tags and values plus a text to stderr ...

it seems to be selected tags only as access and oneway:bicycles are missing for example for the sample above

Way 23681312 [NameCheck=Lippmauer, highway=living_street, lit=yes, mkgmap:admin_level10=Mitte / Hamm-Bossendorf, mkgmap:admin_level2=DEU, mkgmap:admin_level4=Nordrhein-Westfalen, mkgmap:admin_level5=Regierungsbezirk Münster, mkgmap:admin_level6=Kreis Recklinghausen, mkgmap:admin_level8=Haltern am See, mkgmap:postcode=45721, name=Lippmauer, oneway=yes, surface=paving_stones] lines2 checked_element_name=Lippmauer

Hence for now I'm not yet seeing a way to analyze and understand such "virtual" synthesised elements or the state of elements during processing, as I do with OSM elements using JOSM.

GerdP · 2022-06-19 21:09:09

Add a clause like

mkgmap:synthesised=yes {echotags "generated cycleway"}

in lines

argetlam · 2022-06-19 21:14:10

Think, I did, re-using my old statement

highway=* {set NameCheck='Lippmauer'}
name=${NameCheck} { echotags "lines2 checked_element_name=${name}" }

argetlam · 2022-06-19 21:15:15

Although for the original element...

GerdP · 2022-06-19 21:15:57

And you still use --make-opposite-cycleways ?

argetlam · 2022-06-19 21:24:22

No, therefore I just tried to print the tags of the original element (to learn, what the output is), but although the manual said

...all tags...

some where missing.

I'm in the middle of something else atm; will try to come back to this more thoroughly tomorrow.

GerdP · 2022-06-20 06:55:01

Ah, yes, for memory optimization mkgmap ignores all tags which are not used in the style or for internal logic. So, a tag like oneway:bicycle=yes is only stored if your style has a rule that checks the oneway:bicycle=* tag or an option like --make-opposite-cycleways is used which evaluates the tag.

argetlam · 2022-06-20 15:25:40

Oh, clever!

The access tag is used by my style (see line 554 in the opening post for example; it caused the original problems), but not reported for the original element, since it is not set

Way 23681312 [NameCheck=Lippmauer, bicycle=no, highway=living_street, lit=yes, mkgmap:admin_level10=Mitte / Hamm-Bossendorf, mkgmap:admin_level2=DEU, mkgmap:admin_level4=Nordrhein-Westfalen, mkgmap:admin_level5=Regierungsbezirk Münster, mkgmap:admin_level6=Kreis Recklinghausen, mkgmap:admin_level8=Haltern am See, mkgmap:postcode=45721, name=Lippmauer, oneway:bicycle=no, oneway=yes, surface=paving_stones] lines2 checked_element_name=Lippmauer

...for the original element but

Way 23681312 [NameCheck=Lippmauer, access=no, bicycle=yes, highway=living_street, lit=yes, mkgmap:admin_level10=Mitte / Hamm-Bossendorf, mkgmap:admin_level2=DEU, mkgmap:admin_level4=Nordrhein-Westfalen, mkgmap:admin_level5=Regierungsbezirk Münster, mkgmap:admin_level6=Kreis Recklinghausen, mkgmap:admin_level8=Haltern am See, mkgmap:postcode=45721, mkgmap:synthesised=yes, name=Lippmauer, oneway:bicycle=no, oneway=no, surface=paving_stones] lines2 checked_element_name=Lippmauer

for the synthesised copy.

This will serve for simplified orientation for shure.

argetlam · 2022-06-20 17:03:19

Hey Gerd,

would you mind to explain some more to the noob?

Why does

highway=* & osmid()='23681312' { echo "IDmatch checked_element_name=${name}" }

function, but not osmid() used standalone

osmid()='23681312' { echo "IDmatch checked_element_name=${name}" }

This statement is in line 7, but the errors read

SCHWERWIEGEND (global): Error in style: Error: (lines:42): Invalid rule, expression cannot be indexed: osmid()=23681312

Well, 42 is always a good answer
but Lines:42 was

# Hide removed ways

Thanks

//Felix

Last edited by argetlam (2022-06-20 17:04:45)

GerdP · 2022-06-20 17:52:34

That's a rather special problem with the way how mkgmap evaluates tags. Each rule needs at least one clause that checks a tag for existance (a=*) or against a value like with a=xyz or a > 2 or a ~'[0-9]*' . Functions don't work, a!=xyz also doesn't work.
I've never tried to understand the details behind that.

Announcement

#1 2022-06-19 18:52:21

[solved] mkgmap: double processing of ways due to oneway:bicycle=no

#2 2022-06-19 18:54:48

Re: [solved] mkgmap: double processing of ways due to oneway:bicycle=no

#3 2022-06-19 18:59:23

Re: [solved] mkgmap: double processing of ways due to oneway:bicycle=no

#4 2022-06-19 19:02:18

Re: [solved] mkgmap: double processing of ways due to oneway:bicycle=no

#5 2022-06-19 19:14:25

Re: [solved] mkgmap: double processing of ways due to oneway:bicycle=no

#6 2022-06-19 19:30:24

Re: [solved] mkgmap: double processing of ways due to oneway:bicycle=no

#7 2022-06-19 21:02:42

Re: [solved] mkgmap: double processing of ways due to oneway:bicycle=no

#8 2022-06-19 21:09:09

Re: [solved] mkgmap: double processing of ways due to oneway:bicycle=no

#9 2022-06-19 21:14:10

Re: [solved] mkgmap: double processing of ways due to oneway:bicycle=no

#10 2022-06-19 21:15:15

Re: [solved] mkgmap: double processing of ways due to oneway:bicycle=no

#11 2022-06-19 21:15:57

Re: [solved] mkgmap: double processing of ways due to oneway:bicycle=no

#12 2022-06-19 21:24:22

Re: [solved] mkgmap: double processing of ways due to oneway:bicycle=no

#13 2022-06-20 06:55:01

Re: [solved] mkgmap: double processing of ways due to oneway:bicycle=no

#14 2022-06-20 15:25:40

Re: [solved] mkgmap: double processing of ways due to oneway:bicycle=no

#15 2022-06-20 17:03:19

Re: [solved] mkgmap: double processing of ways due to oneway:bicycle=no

#16 2022-06-20 17:52:34

Re: [solved] mkgmap: double processing of ways due to oneway:bicycle=no

Board footer