Your suggestion is probably good, but I have no idea which Windows setting I should check.
I have tried to search for it in the control panel, but can’t find anything about java start-up files.
If you double click on a file in Windows, Windows checks which application is associated to the file type (file-extension). Must be somewhere in your Windows-Settings under “Default apps…” (sorry, don’t have English Windows here). There you should find a list of file-extensions (look for .joz) and the associated default app. I assume .joz-files is associated with the older JOSM-Version still installed somewhere on your system.