Network Rail Data Feeds connection

In my post about Network Rail’s Data Feeds, I outlined the new data feeds that are available for free to developers. I didn’t, however, explain how to connect to them.

The procedure outlined below is in PHP, but the same principles hold across all programming languages. For PHP, I’d recommend using the PECL STOMP package (to install: ‘pecl install stomp’).

<?php
// Basic Network Rail Stomp Handler
$server = "tcp://datafeeds.networkrail.co.uk:61618";
$user = "username";
  // your username at datafeeds.networkrail.co.uk
$password = "password";
  // your password
$channel = "TRAIN_MVT_ALL_TOC";
  // chosen channel (be sure to subscribe via the website)

$con = new Stomp($server, $user, $password);
if (!$con) {
   die('Connection failed: ' . stomp_connect_error());
}
$con->subscribe("/topic/" . $channel);
while($con){
   if ($con->hasFrame()){
      $msg = $con->readFrame();
      foreach (json_decode($msg->body) as $event) {
        // do stuff with $event here
      }
   $con->ack($msg);
   }
}
die('Connection lost: ' . time());
?>

Live rail data from Network Rail

Network Rail hosted a developer conference yesterday regarding open railway data. I attended, along with around 20 others. The movement toward open data has been mostly government directed, aiming to improve data availability and transparency – now Network Rail have been told to get involved.

“We have great trouble working out where the bloody train is” – Network Rail

ATOC (the train operator body, who run National Rail Enquiries) released weekly timetable data in November 2011. It’s this release that allowed me to develop trains.im (in combination with DfT NaPTAN geographical information).

Now, Network Rail is about to release real-time train data feeds. So, what main data is available?

  • Train Movements/TRUST: updates as every scheduled train arrives or departs from a station, or passes a timing point
  • TD: updates on which signal zone every train has just entered (zones typically vary in length from 100 ft to ~2 miles)
  • VSTP: allows for new trains to be scheduled for that day (e.g. to replace a broken down train)

All these are available as live feeds. In short, upon connecting to Network Rail’s servers via. STOMP, messages are continually received indefinitely. There’s about 35,000 scheduled trains a day, all reporting, which means a lot of data to handle.

The feeds will be initially limited to 100 subscribers, and run for at least 3 years.

Unfortunately, there’s lots that’s missing also:

  • CID: alterations to platforms, or changes in stopping pattern when it’s announced (with current feeds, such changes can only be seen as/after they happen) – this information is controlled by ATOC
  • Daily updates to timetables: not currently released in the same format as the weekly ATOC data releases – upon my and other’s requests, Network Rail are exploring its publication
  • Geographical/mileage data: there’s currently no geographical data for anything more than stations – upon my request, Network Rail are exploring its publication
  • Freight trains: both freight trains, and empty/Network Rail trains are deliberately filtered – Network Rail are exploring anonymous freight data, and publication of all other classes
  • Track layout: the issue with the TD signal data is that no-one really knows where these signal zones actually are… - Network Rail are looking into this

What, I think, many people hoped access would be provided to, is an API for ATOCs DARWIN system. This gives every schedule, updates received, ‘official’ predictions, and alteration information. Alas, National Rail Enquiries retain a monopoly here.

All in all, a great effort from Network Rail to promote open-data release (of what they have control over). All in short timescale too – that’s the problem when the Government announce it’ll be out in April.

Vancouver

So, I took the second train of the day from Seattle up to Vancouver, arriving at 11pm on a Friday night. Not exactly the best time to find yourself in a new country. Border control were friendly, and armed with $3 of Canadian money (which I swapped for $3 US on the train): I left the station. Quite like Bristol, the station is nowhere near downtown. Figuring it looked fairly safe, I headed for a bus. Feelings of relative safety and niceness stayed with me even as I walked the last few blocks to the hostel (past several bars). Compared to that time of night in most cities, it was a rather strange experience.

Vancouver is a city which seems to have it all. There’s a vibrant city centre, rocky and sandy beaches, a giant park (with black squirrels!), a river, a market, cycle tracks, mountains. The public transit is very efficient (the skytrain, ferry, and multiple on-time electric and diesel buses) with one ticket covering transfers. The library was built to resemble the Colosseum, and get this: finished on time and on budget. It’s just generally a nice place.

I headed up to Lynn Canyon on a hostel tour, a great park with an old-fashioned suspension bridge. Turns out there was another Bristolian at the hostel: Kate! Canada also has cider (yay!), so we naturally headed out for a few. It seems that not only are apple and pear varieties available, but peach and grapefruit also (certainly sweet). We also both checked out Tim Horton’s for the first time (a Canadian doughnut shop that has since ventured to sandwiches, soups and drinks). Neither of us ended up with quite what we ordered, but it was both very tasty and cheap. Meeting Kate made for a strange reflection: she’d just arrived in Canada to spend a year abroad on a working holiday visa potentially seeking permanent residency, whereas I’d just finished my year abroad albeit in the States with a fixed term. (You can read her rather more poetic account of the tour here.)

Canada also marks the return of Cadbury’s, Nando’s, £1 & £2 coin equivalents, and the Union Jack (it appears surprisingly frequently, at least in British Columbia). Sadly, Canada is also far more expensive than the States – the start of the painful transition to the costly world that is Britain. Such is the pain of living in societies with social care.

I wish I had more time in Vancouver. It’s a really liveable city, and already I want to return.

The rest of western USA

Well, I’m certainly moving fast. There’s a lot I wanted to see on this continent, and I had about 6.5 weeks spare in which to do it. Seems ample? Maybe not.

Yosemite

I headed from San Diego up to Yosemite National Park. The main valley floor is surrounded in nearly all directions by sheer granite rock faces. It’s a really rather spectacular place to visit. I did a couple of the shorter hikes, giving great views over the valley. There were many more far beyond my ability. California is a massively varied state, although equally massive physically.

 

San Francisco

From there I went by train (mostly) to San Francsico. If ever there was a weirdly positioned city, this is it. Nearly all of the downtown streets negotiate huge hills. It’s a nice place, although rain and cold after the continual sun of southern California was a pretty strange experience. Downtown feels really tourist oriented (even if you do nearly trip over the homeless sometimes) which is a bit of a shame really. The whole of the wharf area has been painfully stripped of any character by commercialisation.

I met with Tom Whaples as he was (fortunately) in town. We explored some of the city further from the tourist hub. The ‘Mission’ district is much more the local ‘hipster’ area – with much more of a focus on art and supporting local businesses. There were some neat little stores around there, and a great park with views over the whole of downtown. We also headed out to Glen Park area – the actual park seems to exist because the hill was just too fierce to actually build houses there. Trust me, that’s saying something.

Cable cars (trains pulled by a cable embedded in the ground) continue to run in the city downtown (the last system of its kind!). There’s also a streetcar (tram) system, running underground  in the centre, as well as an overground central line running historic streetcars (from multiple US city networks that have long since been replaced). Add to this electric and diesel buses. It’s a pretty crazy network, it could probably be more efficient. Still, the whole setup is run by the local government and is pretty cheap – something nearly all UK cities could learn from.

The Golden Gate Bridge is truly spectacular. It makes a fun walk, with great views of the city and into the distance. Despite being overcast, I got sunburnt this day. Wholly not impressed.

Then managed to meet with Tom Cairnes (yes, another Tom) due to fortunate timing (he’d just finished a 56-hour train journey, I commenced my own train journey the following day). Was fun. Spent rather a while marvelling at how the electric buses change overhead cables to turn corners (it transpires that in San Francisco the Fahslabend system is used, whereby use of indicators transmits a radio signal to the wire switch).

 

Seattle

So, the train journey up to Seattle took 22 hours. Actually, the ride was fairly entertaining – the continually changing landscape makes for an interesting ride. Heading north, it slowly changed from mountainous to more watery. The ride was stupidly cheap: 908 miles for $110 (£62). (For those strangely interested in statistics, take a look at my train log.) A good way to travel, but even the bus is quicker.

The people of Seattle seem really friendly. It’s a very down-to-earth city, and everyone I encountered was rather helpful. The city itself is a bit more concrete based though. There’s not masses there, but the cute market and random little places make it pretty interesting. The Fresno district has a park at an old gas-works. It gives great views of the city, and it’s a pretty unusual surrounding for random picnic areas. On the subject of food, there’s a well known cuban sandwich shop (Paseo) nearby – and they really are good pork subs.

I stayed out in the ‘International District’ (read: Chinatown). A couple of blocks away was an Asian grocery store (Uwajimaya) featuring a book store (Kinokuniya). I found myself far too entranced there by the various ‘cute’ Japanese artefacts. Never mind the fact I couldn’t read any of the books there – it was a pretty awesome little place.

 

I’ll really miss the States. For all of the failings, it has so much to offer.

And then my next stop: Canada.

SoCal

So, my placement is over. Turned out to be a great 12 months, with fun times both at and outside of work. I’ll surely miss it, waved off to the airport by my housemate and supervisor. Still, everything has to move on. Nearly didn’t make check-in due to terrible traffic on I-95, then airspace closure due to Air Force One landing.

I spent 5 nights over in San Diego, California, a city bordering Mexico. I wish I had more time there – it’s a mid-size city with good variety. The weather is always ‘nice’ – sunny, no clouds, not humid. The bay, on the Pacific Ocean, is actually blue. The coastline and mountains (yes, both) are completely spectacular. Massive roads spiral over bridges between mountainsides. I’ve never had quite so many freckles – obviously I still lack a tan.

It also marked my first time in a hostel. It was pretty fun, sharing a dorm with 2 Northern Irish guys and an Australian. It may have helped that it was voted best hostel in the US though. Given I’ve got about 6 more weeks mostly in hostels, at least I didn’t hate the experience.

Travelling north, I took the Pacific Surfliner train. The scenery was interesting, on a route that spends some of it’s time following the coast, but with average speed of less than 45 mph it’s not the most efficient way to get about.  I stopped overnight in Los Angeles. It’s somewhere I had no real desire to visit, and still don’t. Any good things in LA are outside of LA. The Metro lacks ticket-barriers, instead operating on honesty – seemingly a good idea until you encounter the masses of homelessness there.