Software catalog

Hi guys,

I’m working on OSM Software Catalog. It is improved version of the Software Tables.

Wiki.openstreetmap.org will be used as data source.

My question is how can I integrate it to OSM Wiki?

Catalog will consist of:

  • One HTML file

  • Few Javascript files

  • One JSON file. It will be automatically generated.

I’m just not sure if it can be easily converted to Wiki-doc format.

POC: http://dl.dropbox.com/u/62996911/OpenStreetMap.7z

UPD: Desktop application for Windows https://sourceforge.net/projects/osm-software-catalog/

Before starting coding, I would recommend to test, if existing mediawiki extension already provide the features you would like to have.

For example: On https://www.mediawiki.org/wiki/Extension:TableEdit it’s mentioned that there is some support for http://datatables.net/ .

Thanks. I will take a look.
Looks like I have to find list of supported extensions. And way to contact Wiki administrator to install new extension.

I think you have to setup a test mediawiki and demonstrate that it works and there is an additional benefit. Maybe we should create a copy of the osm wiki where new extensions can be tested.

Regards,
Andreas

List of errors found during processing:
http://dl.dropbox.com/u/62996911/errors.7z

There are a lot of navigateRoute parameters. People do not follow specification.

It’s a wiki. Just like the Openstreetmap Database itself. :wink:

And by the way: Which specification?

Yeah.

A lot of applications. Each line of the “log” file contains name of application, error message related to the parameter, parameter itself and its value.

You might ask user:Tordanik before, he is working on improving his template bot as well. Please stay tuned, as he is currently very busy.

http://dl.dropbox.com/u/62996911/OpenStreetMap.7z
It is first proof-of-concept. Very limited functionality.

I’ve updated archive. Now I’ve achieved my goal to find Android application which supports offline routing.

  1. Filter by Android
  2. Sort by Offline Routing column

:slight_smile:

Sorry, I’d read your question incorrectly. Specification is http://wiki.openstreetmap.org/wiki/Template:Software2

I’ve commented on the topic of unspecified template parameters via mail, but I’d like to describe another idea that may be relevant here.

There are several different projects parsing OSM wiki templates - TTTBot, UserGroupsBot, Taginfo, and an upcoming project by my regional OSM user group, to name only those I have had direct contact to. Therefore, I considered extracting the basic template parsing functionality into a common service serving a json representation of wiki templates.

This service is supposed to offer the following features:

  • make templates more machine-readable by hiding the details of the MediaWiki query api, and filtering out comments and other formatting details from the wiki pages themselves

  • provide a frequently updated view of the wiki by only parsing those pages that have changed since the last run

  • join the translations of a page with templates into a single json object containing all the localised strings

  • support both the case with just one template per page and multiple templates per page

  • provide common functionality such as constructing image thumbnail links, which is necessary for embedding wiki images outside the wiki

TTTBot is still lacking many of those features (and afaik none of the other examples I provided has all of them either), so the software catalogue would also profit from the implementation of this service. But the larger vision is of course to make it easier for all projects to access the OSM wiki, and to avoid reinventing the wheel each time.

To achieve this goal, the service is not supposed to contain any intelligence related to how the template content is being used, that would be the consumers’ task. And it is supposed to require as little configuration as possible: It should figure out the type of the value (text, link, image, url, list, …) from the value itself instead of requiring a full schema for each template.

I’ve even started writing it, but didn’t get very far, and because I’m very busy this month I cannot work on wiki template parsing during that time period at all. I still wanted to to get your feedback on the idea and to ask you to maybe consider it for your future plans.

I like your idea and I’m ready to use your JSON service instead of my JSON generator. We just have to discuss structure of the file.
On my future plans. I’m implementing Catalogue with my colleague. He is busy this week so it will be not so quick as I wish. My vision of this project I tried to explain in my first message of this thread. Other details we can discuss by mail or skype.

Please find my comments below.

If json file would contain all the templates, it will be large. In this case we also have to implement filtering engine which will work on the server side.
Right now we have ~350 apps and size of Software2 template file is 220kb. Not a big but not a small. Gzipped file is 39kb.

Agree.
What if provide link to JSON validator or generator so user will be able to validate his text in the same time and do not wait for bot?

Agree

It will be great. I did not know how to get link to screenshot.

I’m not sure if it is possible. I agree to replace links with “” but lists… Right now items of the list are delimited by semicolon and the Description parameter can have this character too.

It is true, but originally that wasn’t even my intent. What I want to do is making wiki parsing easier. So as a start, one would only replace any code accessing the MediaWiki API and searching for templates in the results with an access to my json representation of wiki templates.

Of course it would be nice if dynamic pages could be generated directly based that json, but as you say this might create additional requirements on the server - such as filtering. Do you think this is possible without semantic knowledge specific to each application?

I’m not sure if I fully understand this. Would you expect the user to paste the template code into that validator, or would that functionality become available somehow after saving the page?

The idea here would be not to decide on one value type for each value, but to include all possible “parsings” for a given value. So in the case of a description with semicolon, you may end up with something like this:


"description" : {
"text": "offline router; good",
"list": ["offline router", "good"]
}

When using that json, the data consumer who thinks/knows that descriptions should be text will look for a “description” that has a “text”, and simply ignores the rest. This moves the knowledge about what value types are expected to the data consumer, allowing the template->json translation to remain generic and easy to extend to more templates.

There are a lot of successful examples: Google Play, AppStore.
Anyway, it is just a database. We have to think how to design structure. To design structure we need list of all use cases and requirements. I will prepare requirements of Catalogue.

We can provide three options:

  • Option to parse Templates like how it works right now. We just have to add some samples and documentation. Maybe implement Template parser in Java?

  • Option to download complete new JSON file. The difference comparing to first option is this file is easy to parse.

  • Option to download part of the new JSON file using server side filtering.

All options will be implemented one by one in the order of the listing. User will choose suitable. It is like .NET/Java approach. You can start at any point of abstraction (File->Stream->Serializer).

I prefer “after saving page”.
If wiki does not support triggers, we can add some button to regenerate JSON file.

I believe it is better to have strong typified fields. If field is of text type, consumer should use it as text. If field is of list type, consumer should treat field as list. Otherwise… it is hard to control software. What we need is template validator (to fix errors in the edit time) and JSON generator with option to generate JSON files of deprecated formats to support old consumers at least some time (in case of format changing). For example right now price field is list. OSM-JSON v1 will have price as list. However I’d like to have separated fields per currency per ApPStore. If we will change JSON structure we still will be able to generate OSM-JSON v1.

I did not find Web developer to implement my idea so I’ve implemented desktop application. Following is a link to download page:

http://sourceforge.net/projects/osm-software-catalog/

Application is still beta. It does not have all possible functionality. I will continue working on this app.

BTW, app can be used as validator for Tables::Software2 tags. It generates log file in the Data folder. There are a lot of warnings.

So are there plans to have OSM Software Catalog also as successor or replacement for the still-out-of-order TTTBot?

I would be happy to make OSM Software Catalog one of the official applications. But I think TTTBot should be fixed too. At least Linux users need it and those who do not want to run any software locally.

There is problem with price field.
0. Some apps are for free

  1. Some applications are for free but some functionality is locked. However it is still possible to use them.
  2. Other applications are for free but they have trial period.
  3. Other applications are for free but maps are not free (or there is download limit)
  4. Some apps are for free for Android but not for iPhone

I need some criterion to determine 0, 1 and 4 apps as freeware… Right now “price” field is plain text field.

v0.0.5 is ready.

  • Settings are persisted
  • New fields are added to filter.
  • Improve controls allocations to fit on small displays
  • Minor changes in the interface
  • Downgrade from .NET 4.5 to .NET 4.0 to be able to run on WinXP