49travel - Travel across Germany with your 49 Euro ticket
This post will be very different than the last few.
Over the last couple of weeks, I have been working on an idea for a webapp.
In May, Germany launched the Deutschland Ticket.
This is an amazing new offer where we can buy a ticket for 49 Euros a month,
and access any public transport across Germany. Its all included!
So, I made a website to find out
all places you could go with this ticket,
how long it takes and why that city is worth visiting.
To start with, it only shows all places you can get to from Hamburg,
but other cities are coming soon. Stay tuned.
Kernel of an idea
The starting idea came during a trip to Bremen. Bremen is just over an hour away from Hamburg and we decided to visit because of the freedom of the 49 Euro ticket. You can now visit so many places without worrying about the cost of the travel. Bremen was great but then we started thinking. How do we find out ALL places we could visit that we may not know about? And that was the starting point. I started with the notion that this should be fairly simple to do. Deutsch Bahn should have some API and we simply need to extract data from that, right?
API shenanigans
So I registered for DB API’s. And I tried to follow the documentation. And zilch. There is more or less just one API and all it tells me are the timetables at particular stations. It does not help since what I wanted to get were journeys between stations. I deep dived into the world of available API’s and found an amazing resource. There’s a Transport API, that actually has the API that, in its own words “it returns whatever data the DB Navigator app shows”. That is amazing! So we have an API.
What’s the content?
So we have an API, but what do we get out of it? Of course, first we use query parameters to exclude IC/ICE/EC trains, since they are not included in the 49 euro ticket. Weirdly the API considers Flix trains as regional trains so I had to filter them out.
The very first requirement I had put when starting to work on this
was that I want to know all places I could get to from Hamburg.
So, my initial idea was to just get all “Trips” from Hamburg.
This would mean that I find all destinations of regional trains.
And then the destinations after that.
Immediately, that notion had 2 problems.
The Trips endpoint does not provide journey times,
so I would have to call yet another API
to find journey times.
Second, and more important, I would miss places that are not destinations.
This immediately excludes places like Lübeck which is a popular tourist destination.
So, I needed to rethink.
WikiVoyage
Clearly, the answer was that we first need to find the places to visit,
and then do a Journey query from Hamburg to that destination.
But where do we find those places.
That’s where I started thinking about WikiVoyage.
I knew it existed, but does it have an API or would I have to scrape it?
And how would I know how to extract touristy places?
Well, first, it turns out WikiVoyage has the exact same API as Wikipedia!
So, you use it the exact same way and just replace wikipedia with wikivoyage
in the URL.
And you can just use the API to get page extracts for the full page.
But what about finding cities.
After some furious Googling, I found, as is typical, some random post
on StackOverflow that mentions Categories.
Category pages have the directory structure of a particular category.
So everything belonging to some category will have a top page,
and then sub-categories and then finally all pages within those category and sub-categories.
So of course, there is a Category: Germany page.
And so I used a queue to traverse the whole directory structure and get all cities.
There was some filtering required because some of the pages were
“Regions” and not “Cities” but that was fairly easy.
Once I had all cities(or towns), I used the tranport API to extract
Stop ID for each city.
And then I could just do a Journey query between Hamburg and those cities.
So, I could extract journey times for all of those cities.
Now what?
Deploy
From the start I felt that this was going to be the scary part. I have no frontend experience besides using Streamlit or Dash, which would be insufficient. I was sure I needed a frontend that made it useful, but how do I do that with my limited knowledge. So, I had to make a leap of faith. I knew that Vercel makes it easy to deploy websites. So I made an account and staying to true to engineering wisdom, made a boring choice. I deployed the Create React App on Vercel. The leap of faith was that with zero JavaScript experience I could still make it work with ChatGPT. But I felt a bit overwhelmed and so decided that the first version was going to be just a big MarkDown file, since MarkDown I understood. So that was my first version. A huge text file! On the internet.
Enter gpt-3.5-turbo
I now had a trip API and all cities worth visiting in Germany.
But what do I put on the website?
My initial idea was that I would put the list of cities,
sorted by journey time and a short touristy description for the city.
But where do I get the description?
Initially I thought I would just scrape WikiVoyage.
But the text was very unstructured and I get very weird output.
The solution, and it took some time for me to figure this out,
was to actually use gpt-3.5-turbo to summarize the pages.
I will write another post about that since I had to do quite a bit of experimentation for that.
That was the only part that cost money. A whopping $4 for summarizing ALL cities
in Germany on Wikivoyage!
Share app and brace for feedback
So, having finally created a crazy big text file and put it on the internet, I decided to share it with friends. I was sure that the ugliness would be the first thing that would be commented on. But there were quite a few suggestions apart from the ugliness. Infinite scroll was one of the ones I was surprised about. Another one that I had not considered were having live status(solving that was another rabbit hole).
Next I went on another quest of fighting with React using ChatGPT. I made the wrong choice of using Material UI since I had read somewhere that it was the popular choice. That was a mistake. Chose boring technology everytime. I realized my mistake and shifted to Bootstrap. And gradually made some progress. Decided on a simple layout and organized the pages. Put in drop down buttons. And finally, pagination to remove infinite scroll. And that’s what you see now.
Final thoughts
This was a long post. And while writing it, I realized that there was so much that I skimmed over. It was a fun build, but that does not mean I am going to stop. The initial objective was to make something quickly, ask for feedback and iterate. I think I accomplished that. There’s more to do, but I think its at a stage where I can share it with the world. Hopefully, you find it useful. I know I will.
