merge master into next

docs/changelog.md

@@ -0,0 +1,6 @@
+# Changelog
+
+## `2.5.0`
+
+- new [Schleswig-Holstein (NAH.SH)](https://de.wikipedia.org/wiki/Nahverkehrsverbund_Schleswig-Holstein) [profile](../p/nahsh)
+- new [*writing a profile* guide](./writing-a-profile.md)

docs/readme.md

@@ -0,0 +1,13 @@
+# API documentation
+
+- [`journeys(from, to, [opt])`](journeys.md) – get journeys between locations
+- [`journeyLeg(ref, lineName, [opt])`](journey-leg.md) – get details for a leg of a journey
+- [`departures(station, [opt])`](departures.md) – query the next departures at a station
+- [`locations(query, [opt])`](locations.md) – find stations, POIs and addresses
+- [`location(id)`](location.md) – get details about a location
+- [`nearby(location, [opt])`](nearby.md) – show stations & POIs around
+- [`radar(north, west, south, east, [opt])`](radar.md) – find all vehicles currently in a certain area
+
+## Writing a profile
+
+Check [the guide](writing-a-profile.md).

docs/writing-a-profile.md

@@ -0,0 +1,166 @@
+# Writing a profile
+
+**Per endpoint, `hafas-client` has an endpoint-specific customisation called *profile*** which may for example do the following:
+
+- handle the additional requirements of the endpoint (e.g. authentication),
+- extract additional information from the data provided by the endpoint,
+- guard against triggering bugs of certain endpoints (e.g. time limits).
+
+This guide is about writing such a profile. If you just want to use an already supported endpoint, refer to the [API documentation](readme.md) instead.
+
+*Note*: **If you get stuck, ask for help by [creating an issue](https://github.com/derhuerst/hafas-client/issues/new)!** We want to help people expand the scope of this library.
+
+## 0. How do the profiles work?
+
+A profile contains of three things:
+
+- **mandatory details about the HAFAS endpoint**
+	- `endpoint`: The protocol, host and path of the endpoint.
+	- `locale`: The [BCP 47](https://en.wikipedia.org/wiki/IETF_language_tag) [locale](https://en.wikipedia.org/wiki/Locale_(computer_software)) of your endpoint (or the area that your endpoint covers).
+	- `timezone`: An [IANA-time-zone](https://www.iana.org/time-zones)-compatible [timezone](https://en.wikipedia.org/wiki/Time_zone) of your endpoint.
+- **flags indicating that features are supported by the endpoint** – e.g. `journeyRef`
+- **methods overriding the [default profile](../lib/default-profile.js)**
+
+As an example, let's say we have an [Austrian](https://en.wikipedia.org/wiki/Austria) endpoint:
+
+```js
+const myProfile = {
+	endpoint: 'https://example.org/bin/mgate.exe',
+	locale: 'de-AT',
+	timezone: 'Europe/Vienna'
+}
+```
+
+Assuming the endpoint returns all lines names prefixed with `foo `, We can strip them like this:
+
+```js
+// get the default line parser
+const createParseLine = require('hafas-client/parse/line')
+
+const createParseLineWithoutFoo = (profile, operators) => {
+	const parseLine = createParseLine(profile, operators)
+
+	// wrapper function with additional logic
+	const parseLineWithoutFoo = (l) => {
+		const line = parseLine(l)
+		line.name = line.name.replace(/foo /g, '')
+		return line
+	}
+	return parseLineWithoutFoo
+}
+
+profile.parseLine = createParseLineWithoutFoo
+```
+
+If you pass this profile into `hafas-client`, the `parseLine` method will override [the default one](../parse/line.js).
+
+## 1. Setup
+
+*Note*: There are many ways to find the required values. This way is rather easy and has worked for most of the apps that we've looked at so far.
+
+1. **Get an iOS or Android device and download the "official" app** for the public transport provider that you want to build a profile for.
+2. **Configure a [man-in-the-middle HTTP proxy](https://docs.mitmproxy.org/stable/concepts-howmitmproxyworks/)** like [mitmproxy](https://mitmproxy.org).
+	- Configure your device to trust the self-signed SSL certificate, [as outlined in the mitmproxy docs](https://docs.mitmproxy.org/stable/concepts-certificates/).
+	- *Note*: This method does not work if the app uses [public key pinning](https://en.wikipedia.org/wiki/HTTP_Public_Key_Pinning). In this case (the app won't be able to query data), please [create an issue](https://github.com/derhuerst/hafas-client/issues/new), so we can discuss other techniques.
+3. **Record requests of the app.**
+	- [There's a video showing this step](https://stuff.jannisr.de/how-to-record-hafas-requests.mp4).
+	- Make sure to cover all relevant sections of the app, e.g. "journeys", "departures", "live map". Better record more than less; You will regret not having enough information later on.
+	- To help others in the future, post the requests (in their entirety!) on GitHub, e.g. in as format like [this](https://gist.github.com/derhuerst/5fa86ed5aec63645e5ae37e23e555886). This will also let us help you if you have any questions.
+
+## 2. Basic profile
+
+- **Identify the `endpoint`.** The protocol, host and path of the endpoint, *but not* the query string.
+	- *Note*: **`hafas-client` for now only supports the interface providing JSON** (generated from XML), which is being used by the corresponding iOS/Android apps. It supports neither the JSONP, nor the XML, nor the HTML interface. If the endpoint does not end in `mgate.exe`, it mostly likely won't work.
+- **Identify the `locale`.** Basically guess work; Use the date & time formats as an indicator.
+- **Identify the `timezone`.** This may be tricky, a for example [Deutsche Bahn](https://en.wikipedia.org/wiki/Deutsche_Bahn) returns departures for Moscow as `+01:00` instead of `+03:00`.
+- **Copy the authentication** and other meta fields, namely `ver`, `ext`, `client` and `lang`.
+	- You can find these fields in the root of each request JSON. Check [a VBB request](https://gist.github.com/derhuerst/5fa86ed5aec63645e5ae37e23e555886#file-1-http-L13-L22) and [the corresponding VBB profile](https://github.com/derhuerst/hafas-client/blob/6e61097687a37b60d53e767f2711466b80c5142c/p/vbb/index.js#L22-L29) for an example.
+	- Add a function `transformReqBody(body)` to your profile, which assigns them to `body`.
+	- Some profiles have a `checksum` parameter (like [here](https://gist.github.com/derhuerst/2a735268bd82a0a6779633f15dceba33#file-journey-details-1-http-L1)) or two `mic` & `mac` parameters (like [here](https://gist.github.com/derhuerst/5fa86ed5aec63645e5ae37e23e555886#file-1-http-L1)). If you see one of them in your requests, jump to [*Appendix A: checksum, mic, mac*](#appendix-a-checksum-mic-mac). Unfortunately, this is necessary to get the profile working.
+
+## 3. Products
+
+In `hafas-client`, there's a difference between the `mode` and the `product` field:
+
+- The `mode` field describes the mode of transport in general. [Standardised by the *Friendly Public Transport Format* `1.0.1`](https://github.com/public-transport/friendly-public-transport-format/blob/1.0.1/spec/readme.md#modes), it is on purpose limited to a very small number of possible values, e.g. `train` or `bus`.
+- The value for `product` relates to how a means of transport "works" *in local context*. Example: Even though [*S-Bahn*](https://en.wikipedia.org/wiki/Berlin_S-Bahn) and [*U-Bahn*](https://en.wikipedia.org/wiki/Berlin_U-Bahn) in Berlin are both `train`s, they have different operators, service patterns, stations and look different. Therefore, they are two distinct `product`s `subway` and `suburban`.
+
+**Specify `product`s that appear in the app** you recorded requests of. For a fictional transit network, this may look like this:
+
+```js
+const products = {
+	commuterTrain: {
+		product: 'commuterTrain',
+		mode: 'train',
+		bitmask: 1,
+		name: 'ACME Commuter Rail',
+		short: 'CR'
+	},
+	metro: {
+		product: 'metro',
+		mode: 'train',
+		bitmask: 2,
+		name: 'Foo Bar Metro',
+		short: 'M'
+	}
+}
+```
+
+Let's break this down:
+
+- `product`: A sensible, [camelCased](https://en.wikipedia.org/wiki/Camel_case#Variations_and_synonyms), alphanumeric identifier. Use it for the key in the `products` object as well.
+- `mode`: A [valid *Friendly Public Transport Format* `1.0.1` mode](https://github.com/public-transport/friendly-public-transport-format/blob/1.0.1/spec/readme.md#modes).
+- `bitmask`: HAFAS endpoints work with a [bitmask](https://en.wikipedia.org/wiki/Mask_(computing)#Arguments_to_functions) that toggles the individual products. the value should toggle the appropriate bit(s) in the bitmask (see below).
+- `name`: A short, but distinct name for the means of transport, *just precise enough in local context*, and in the local language. In Berlin, `S-Bahn-Schnellzug` would be too much, because everyone knows what `S-Bahn` means.
+- `short`: The shortest possible symbol that identifies the product.
+
+todo: `defaultProducts`, `allProducts`, `bitmasks`, add to profile
+
+If you want, you can now **verify that the profile works**; We've prepared [a script](https://runkit.com/derhuerst/hafas-client-profile-example) for that. Alternatively, [submit a Pull Request](https://help.github.com/articles/creating-a-pull-request-from-a-fork/) and we will help you out with testing and improvements.
+
+### Finding the right values for the `bitmask` field
+
+As shown in [the video](https://stuff.jannisr.de/how-to-record-hafas-requests.mp4), search for a journey and toggle off one product at a time, recording the requests. After extracting the products bitmask ([example](https://gist.github.com/derhuerst/193ef489f8aa50c2343f8bf1f2a22069#file-via-http-L34)) you will end up with values looking like these:
+
+```
+toggles                     value  binary  subtraction     bit(s)
+all products                31     11111   31 - 0
+all but ACME Commuter Rail  15     01111   31 - 2^4        2^4
+all but Foo Bar Metro       23     10111   31 - 2^3        2^3
+all but product E           30     11001   31 - 2^2 - 2^1  2^2, 2^1
+all but product F           253    11110   31 - 2^1        2^0
+```
+
+## 4. Additional info
+
+We consider these improvements to be *optional*:
+
+- **Check if the endpoint supports the journey legs call.**
+	- In the app, check if you can query details for the status of a single journey leg. It should load realtime delays and the current progress.
+	- If this feature is supported, add `journeyLeg: true` to the profile.
+- **Check if the endpoint supports the live map call.** Does the app have a "live map" showing all vehicles within an area? If so, add `radar: true` to the profile.
+-  **Consider transforming station & line names** into the formats that's most suitable for *local users*. Some examples:
+	- `M13 (Tram)` -> `M13`. With Berlin context, it is obvious that `M13` is a tram.
+	- `Berlin Jungfernheide Bhf` -> `Berlin Jungfernheide`. With local context, it's obvious that *Jungfernheide* is a train station.
+- **Check if the endpoint has non-obvious limitations** and let use know about these. Examples:
+	- Some endpoints have a time limit, after which they won't return more departures, but silently discard them.
+
+---
+
+## Appendix A: `checksum`, `mic`, `mac`
+
+As far as we know, there are three different types of authentication used among HAFAS deployments.
+
+### unprotected endpoints
+
+You can just query these, as long as you send a formally correct request.
+
+### endpoints using the `checksum` query parameter
+
+`checksum` is a [message authentication code](https://en.wikipedia.org/wiki/Message_authentication_code): `hafas-client` will compute it by [hashing](https://en.wikipedia.org/wiki/Hash_function) the request body and a secret *salt*. **This secret can be read from the config file inside the app bundle.** There is no guide for this yet, so please [open an issue](https://github.com/derhuerst/hafas-client/issues/new) instead.
+
+### endpoints using the `mic` & `mac` query parameters
+
+`mic` is a [message integrity code](https://en.wikipedia.org/wiki/Message_authentication_code), the [hash](https://en.wikipedia.org/wiki/Hash_function) of the request body.
+
+`mac` is a [message authentication code](https://en.wikipedia.org/wiki/Message_authentication_code), the hash of `mic` and a secret *salt*. **This secret can be read from the config file inside the app bundle.** There is no guide for this yet, so please [open an issue](https://github.com/derhuerst/hafas-client/issues/new) instead.

lib/default-profile.js

@@ -59,7 +59,7 @@ const defaultProfile = {
 	formatRectangle,
 	filters,
 
-	journeysNumF: true, // `journeys()` method: support for `numF` field
+	journeysNumF: true, // `journeys()` method: support for `numF` field?
 	journeyLeg: false,
 	radar: false
 }

p/nahsh/example.js

@@ -0,0 +1,20 @@
+'use strict'
+
+const createClient = require('../..')
+const nahshProfile = require('.')
+
+const client = createClient(nahshProfile)
+
+// Flensburg Hbf to Kiel Hbf
+client.journeys('8000103', '8000199', {results: 10, tickets: true})
+// client.departures('8000199', {duration: 10})
+// client.journeyLeg('1|30161|5|100|14032018', 'Bus 52')
+// client.locations('Schleswig', {results: 1})
+// client.location('706990') // Kiel Holunderbusch
+// client.nearby({type: 'location', latitude: 54.295691, longitude: 10.116424}, {distance: 60})
+// client.radar(54.4, 10.0, 54.2, 10.2, {results: 10})
+
+.then((data) => {
+	console.log(require('util').inspect(data, {depth: null}))
+})
+.catch(console.error)

p/nahsh/index.js

@@ -0,0 +1,144 @@
+'use strict'
+
+const createParseBitmask = require('../../parse/products-bitmask')
+const createFormatBitmask = require('../../format/products-bitmask')
+const _createParseLine = require('../../parse/line')
+const _parseLocation = require('../../parse/location')
+const _createParseJourney = require('../../parse/journey')
+
+const products = require('./products')
+
+// todo: journey prices
+
+const transformReqBody = (body) => {
+	body.client = {
+		id: 'NAHSH',
+		name: 'NAHSHPROD',
+		v: '3000700'
+	}
+	body.ver = '1.16'
+	body.auth = {aid: 'r0Ot9FLFNAFxijLW'}
+	body.lang = 'de'
+
+	return body
+}
+
+const parseLocation = (profile, l, lines) => {
+	const res = _parseLocation(profile, l, lines)
+	// weird fix for empty lines, e.g. IC/EC at Flensburg Hbf
+	if (res.lines) {
+		res.lines = res.lines.filter(x => x.id && x.name)
+	}
+
+	// remove leading zeroes, todo
+	if (res.id && res.id.length > 0) {
+		res.id = res.id.replace(/^0+/, '')
+	}
+
+	return res
+}
+
+const createParseLine = (profile, operators) => {
+	const parseLine = _createParseLine(profile, operators)
+
+	const parseLineWithMode = (l) => {
+		const res = parseLine(l)
+
+		res.mode = res.product = null
+		if ('class' in res) {
+			const data = products.bitmasks[parseInt(res.class)]
+			if (data) {
+				res.mode = data.mode
+				res.product = data.product
+			}
+		}
+
+		return res
+	}
+	return parseLineWithMode
+}
+
+const createParseJourney = (profile, stations, lines, remarks) => {
+	const parseJourney = _createParseJourney(profile, stations, lines, remarks)
+
+	const parseJourneyWithTickets = (j) => {
+		const res = parseJourney(j)
+
+		if (
+			j.trfRes &&
+			Array.isArray(j.trfRes.fareSetL) &&
+			j.trfRes.fareSetL.length > 0
+		) {
+			res.tickets = []
+
+			for (let t of j.trfRes.fareSetL) {
+				const tariff = t.desc
+				if (!tariff || !Array.isArray(t.fareL)) continue
+				for (let v of t.fareL) {
+					const variant = v.name
+					if(!variant) continue
+					const ticket = {
+						name: [tariff, variant].join(' - '),
+						tariff,
+						variant
+					}
+					if (v.prc && Number.isInteger(v.prc) && v.cur) {
+						ticket.amount = v.prc/100
+						ticket.currency = v.cur
+					} else {
+						ticket.amount = null
+						ticket.hint = 'No pricing information available.'
+					}
+					res.tickets.push(ticket)
+				}
+			}
+		}
+
+		return res
+	}
+
+	return parseJourneyWithTickets
+}
+
+const defaultProducts = {
+	nationalExp: true,
+	national: true,
+	interregional: true,
+	regional: true,
+	suburban: true,
+	bus: true,
+	ferry: true,
+	subway: true,
+	tram: true,
+	onCall: true
+}
+const formatBitmask = createFormatBitmask(products)
+const formatProducts = (products) => {
+	products = Object.assign(Object.create(null), defaultProducts, products)
+	return {
+		type: 'PROD',
+		mode: 'INC',
+		value: formatBitmask(products) + ''
+	}
+}
+
+const nahshProfile = {
+	locale: 'de-DE',
+	timezone: 'Europe/Berlin',
+	endpoint: 'http://nah.sh.hafas.de/bin/mgate.exe',
+	transformReqBody,
+
+	products: products.allProducts,
+
+	parseProducts: createParseBitmask(products.allProducts, defaultProducts),
+	parseLine: createParseLine,
+	parseLocation,
+	parseJourney: createParseJourney,
+
+	formatProducts,
+
+	journeyLeg: true,
+	radar: false // todo: see #34
+}
+
+module.exports = nahshProfile

p/nahsh/products.js

@@ -0,0 +1,101 @@
+'use strict'
+
+const p = {
+	nationalExp: {
+		bitmask: 1,
+		name: 'High-speed rail',
+		short: 'ICE/HSR',
+		mode: 'train',
+		product: 'nationalExp'
+	},
+	national: {
+		bitmask: 2,
+		name: 'InterCity & EuroCity',
+		short: 'IC/EC',
+		mode: 'train',
+		product: 'national'
+	},
+	interregional: { // todo: also includes EN?
+		bitmask: 4,
+		name: 'Interregional',
+		short: 'IR',
+		mode: 'train',
+		product: 'interregional'
+	},
+	regional: {
+		bitmask: 8,
+		name: 'Regional & RegionalExpress',
+		short: 'RB/RE',
+		mode: 'train',
+		product: 'regional'
+	},
+	suburban: {
+		bitmask: 16,
+		name: 'S-Bahn',
+		short: 'S',
+		mode: 'train',
+		product: 'suburban'
+	},
+	bus: {
+		bitmask: 32,
+		name: 'Bus',
+		short: 'B',
+		mode: 'bus',
+		product: 'bus'
+	},
+	ferry: {
+		bitmask: 64,
+		name: 'Ferry',
+		short: 'F',
+		mode: 'watercraft',
+		product: 'ferry'
+	},
+	subway: {
+		bitmask: 128,
+		name: 'U-Bahn',
+		short: 'U',
+		mode: 'train',
+		product: 'subway'
+	},
+	tram: {
+		bitmask: 256,
+		name: 'Tram',
+		short: 'T',
+		mode: 'train',
+		product: 'tram'
+	},
+	onCall: {
+		bitmask: 512,
+		name: 'On-call transit',
+		short: 'on-call',
+		mode: null, // todo
+		product: 'onCall'
+	}
+}
+
+p.bitmasks = []
+p.bitmasks[1] = p.nationalExp
+p.bitmasks[2] = p.national
+p.bitmasks[4] = p.interregional
+p.bitmasks[8] = p.regional
+p.bitmasks[16] = p.suburban
+p.bitmasks[32] = p.bus
+p.bitmasks[64] = p.ferry
+p.bitmasks[128] = p.subway
+p.bitmasks[256] = p.tram
+p.bitmasks[512] = p.onCall
+
+p.allProducts = [
+	p.nationalExp,
+	p.national,
+	p.interregional,
+	p.regional,
+	p.suburban,
+	p.bus,
+	p.ferry,
+	p.subway,
+	p.tram,
+	p.onCall
+]
+
+module.exports = p

p/nahsh/readme.md

@@ -0,0 +1,18 @@
+# NAH.SH profile for `hafas-client`
+
+[*Nahverkehrsverbund Schleswig-Holstein (NAH.SH)*](https://de.wikipedia.org/wiki/Nahverkehrsverbund_Schleswig-Holstein) is the transportation authority for regional transport in [Schleswig-Holstein](https://en.wikipedia.org/wiki/Schleswig-Holstein). This profile adds *NAH.SH*-specific customizations to `hafas-client`. Consider using [`nahsh-hafas`](https://github.com/juliuste/nahsh-hafas), to always get the customized client right away.
+
+## Usage
+
+```js
+const createClient = require('hafas-client')
+const nahshProfile = require('hafas-client/p/nahsh')
+
+// create a client with NAH.SH profile
+const client = createClient(nahshProfile)
+```
+
+
+## Customisations
+
+- parses *NAH.SH*-specific products

p/vbb/index.js

@@ -9,7 +9,6 @@ const getStations = require('vbb-stations')
 const _createParseLine = require('../../parse/line')
 const _parseLocation = require('../../parse/location')
 const _createParseJourney = require('../../parse/journey')
-const _createParseStopover = require('../../parse/stopover')
 const _createParseDeparture = require('../../parse/departure')
 const _formatStation = require('../../format/station')
 
@@ -86,20 +85,6 @@ const createParseJourney = (profile, stations, lines, remarks) => {
 	return parseJourneyWithTickets
 }
 
-const createParseStopover = (profile, stations, lines, remarks, connection) => {
-	const parseStopover = _createParseStopover(profile, stations, lines, remarks, connection)
-
-	const parseStopoverWithShorten = (st) => {
-		const res = parseStopover(st)
-		if (res.station && res.station.name) {
-			res.station.name = shorten(res.station.name)
-		}
-		return res
-	}
-
-	return parseStopoverWithShorten
-}
-
 const createParseDeparture = (profile, stations, lines, remarks) => {
 	const parseDeparture = _createParseDeparture(profile, stations, lines, remarks)
 
@@ -152,7 +137,6 @@ const vbbProfile = {
 	parseLine: createParseLine,
 	parseJourney: createParseJourney,
 	parseDeparture: createParseDeparture,
-	parseStopover: createParseStopover,
 
 	formatStation,
 

package.json

@@ -1,7 +1,7 @@
 {
 	"name": "hafas-client",
 	"description": "JavaScript client for HAFAS public transport APIs.",
-	"version": "2.4.0",
+	"version": "2.5.0",
 	"main": "index.js",
 	"files": [
 		"index.js",

parse/departure.js

@@ -22,14 +22,14 @@ const createParseDeparture = (profile, stations, lines, remarks) => {
 		}
 		// todo: res.trip from rawLine.prodCtx.num?
 
+		// todo: DRY with parseStopover
+		// todo: DRY with parseJourneyLeg
 		if (d.stbStop.dTimeR && d.stbStop.dTimeS) {
 			const realtime = profile.parseDateTime(profile, d.date, d.stbStop.dTimeR)
 			const planned = profile.parseDateTime(profile, d.date, d.stbStop.dTimeS)
 			res.delay = Math.round((realtime - planned) / 1000)
 		} else res.delay = null
 
-		// todo: follow public-transport/friendly-public-transport-format#27 here
-		// see also derhuerst/vbb-rest#19
 		if (d.stbStop.aCncl || d.stbStop.dCncl) {
 			res.cancelled = true
 			Object.defineProperty(res, 'canceled', {value: true})

parse/journey-leg.js

@@ -47,7 +47,7 @@ const createParseJourneyLeg = (profile, stations, lines, remarks) => {
 			if (pt.arr.aPlatfS) res.arrivalPlatform = pt.arr.aPlatfS
 
 			if (passed && pt.jny.stopL) {
-				const parse = profile.parseStopover(profile, stations, lines, remarks, j)
+				const parse = profile.parseStopover(profile, stations, lines, remarks, j.date)
 				const passedStations = pt.jny.stopL.map(parse)
 				// filter stations the train passes without stopping, as this doesn't comply with fptf (yet)
 				res.passed = passedStations.filter((x) => !x.passBy)
@@ -71,22 +71,22 @@ const createParseJourneyLeg = (profile, stations, lines, remarks) => {
 			}
 		}
 
-		// todo: follow public-transport/friendly-public-transport-format#27 here
+		// todo: DRY with parseDeparture
-		// see also derhuerst/vbb-rest#19
+		// todo: DRY with parseStopover
-		if (pt.arr.aCncl) {
+		if (pt.arr.aCncl || pt.dep.dCncl) {
 			res.cancelled = true
 			Object.defineProperty(res, 'canceled', {value: true})
+			if (pt.arr.aCncl) {
 				res.arrival = res.arrivalPlatform = res.arrivalDelay = null
 				const arr = profile.parseDateTime(profile, j.date, pt.arr.aTimeS)
 				res.formerScheduledArrival = arr.toISO()
 			}
 			if (pt.dep.dCncl) {
-			res.cancelled = true
-			Object.defineProperty(res, 'canceled', {value: true})
 				res.departure = res.departurePlatform = res.departureDelay = null
 				const dep = profile.parseDateTime(profile, j.date, pt.dep.dTimeS)
 				res.formerScheduledDeparture = dep.toISO()
 			}
+		}
 
 		return res
 	}

parse/location.js

@@ -19,7 +19,7 @@ const parseLocation = (profile, l, lines) => {
 		const station = {
 			type: 'station',
 			id: l.extId,
-			name: l.name,
+			name: profile.parseStationName(l.name),
 			location: 'number' === typeof res.latitude ? res : null
 		}
 

parse/movement.js

@@ -9,32 +9,7 @@ const createParseMovement = (profile, locations, lines, remarks) => {
 	// todo: what is m.ani.proc[n]? wut?
 	// todo: how does m.ani.poly work?
 	const parseMovement = (m) => {
-		const parseNextStop = (s) => {
+		const pStopover = profile.parseStopover(profile, locations, lines, remarks, m.date)
-			const dep = s.dTimeR || s.dTimeS
-				? profile.parseDateTime(profile, m.date, s.dTimeR || s.dTimeS)
-				: null
-			const arr = s.aTimeR || s.aTimeS
-				? profile.parseDateTime(profile, m.date, s.aTimeR || s.aTimeS)
-				: null
-
-			const res = {
-				station: locations[s.locX],
-				departure: dep ? dep.toISO() : null,
-				arrival: arr ? arr.toISO() : null
-			}
-
-			if (m.dTimeR && m.dTimeS) {
-				const plannedDep = profile.parseDateTime(profile, m.date, s.dTimeS)
-				res.departureDelay = Math.round((dep - plannedDep) / 1000)
-			} else res.departureDelay = null
-
-			if (m.aTimeR && m.aTimeS) {
-				const plannedArr = profile.parseDateTime(profile, m.date, s.aTimeS)
-				res.arrivalDelay = Math.round((arr - plannedArr) / 1000)
-			} else res.arrivalDelay = null
-
-			return res
-		}
 
 		const res = {
 			direction: profile.parseStationName(m.dirTxt),
@@ -45,7 +20,7 @@ const createParseMovement = (profile, locations, lines, remarks) => {
 				latitude: m.pos.y / 1000000,
 				longitude: m.pos.x / 1000000
 			} : null,
-			nextStops: m.stopL.map(parseNextStop),
+			nextStops: m.stopL.map(pStopover),
 			frames: []
 		}
 

parse/products-bitmask.js

@@ -19,6 +19,9 @@ const createParseBitmask = (profile) => {
 				res[product.id] = true
 				bitmask -= pBitmask
 			}
+			else{
+				res[product.product] = false
+			}
 		}
 
 		return res

parse/stopover.js

@@ -2,32 +2,38 @@
 
 // todo: arrivalDelay, departureDelay or only delay ?
 // todo: arrivalPlatform, departurePlatform
-const createParseStopover = (profile, stations, lines, remarks, connection) => {
+const createParseStopover = (profile, stations, lines, remarks, date) => {
 	const parseStopover = (st) => {
 		const res = {
 			station: stations[parseInt(st.locX)] || null
 		}
 		if (st.aTimeR || st.aTimeS) {
-			const arr = profile.parseDateTime(profile, connection.date, st.aTimeR || st.aTimeS)
+			const arr = profile.parseDateTime(profile, date, st.aTimeR || st.aTimeS)
 			res.arrival = arr.toISO()
 		}
 		if (st.dTimeR || st.dTimeS) {
-			const dep = profile.parseDateTime(profile, connection.date, st.dTimeR || st.dTimeS)
+			const dep = profile.parseDateTime(profile, date, st.dTimeR || st.dTimeS)
 			res.departure = dep.toISO()
 		}
 
 		// mark stations the train passes without stopping
 		if(st.dInS === false && st.aOutS === false) res.passBy = true
 
-		// todo: follow public-transport/friendly-public-transport-format#27 here
+		// todo: DRY with parseDeparture
-		// see also derhuerst/vbb-rest#19
+		// todo: DRY with parseJourneyLeg
-		if (st.aCncl) {
+		if (st.aCncl || st.dCncl) {
 			res.cancelled = true
-			res.arrival = null
+			Object.defineProperty(res, 'canceled', {value: true})
+			if (st.aCncl) {
+				res.arrival = res.arrivalDelay = null
+				const arr = profile.parseDateTime(profile, d.date, st.aTimeS)
+				res.formerScheduledArrival = arr.toISO()
 			}
 			if (st.dCncl) {
-			res.cancelled = true
+				res.departure = res.departureDelay = null
-			res.departure = null
+				const arr = profile.parseDateTime(profile, d.date, st.dTimeS)
+				res.formerScheduledDeparture = arr.toISO()
+			}
 		}
 
 		return res

readme.md

@@ -4,10 +4,11 @@
 
 HAFAS endpoint | wrapper library | docs | example code | source code
 ---------------|------------------|------|---------|------------
-[Deutsche Bahn](https://en.wikipedia.org/wiki/Deutsche_Bahn) | [`db-hafas`](https://github.com/derhuerst/db-hafas) | [docs](p/db/readme.md) | [example code](p/db/example.js) | [src](p/db/index.js)
+[Deutsche Bahn (DB)](https://en.wikipedia.org/wiki/Deutsche_Bahn) | [`db-hafas`](https://github.com/derhuerst/db-hafas) | [docs](p/db/readme.md) | [example code](p/db/example.js) | [src](p/db/index.js)
-[Berlin & Brandenburg public transport](https://en.wikipedia.org/wiki/Verkehrsverbund_Berlin-Brandenburg) | [`vbb-hafas`](https://github.com/derhuerst/vbb-hafas) | [docs](p/vbb/readme.md) | [example code](p/vbb/example.js) | [src](p/vbb/index.js)
+[Berlin & Brandenburg public transport (VBB)](https://en.wikipedia.org/wiki/Verkehrsverbund_Berlin-Brandenburg) | [`vbb-hafas`](https://github.com/derhuerst/vbb-hafas) | [docs](p/vbb/readme.md) | [example code](p/vbb/example.js) | [src](p/vbb/index.js)
 [Österreichische Bundesbahnen (ÖBB)](https://en.wikipedia.org/wiki/Austrian_Federal_Railways) | [`oebb-hafas`](https://github.com/juliuste/oebb-hafas) | [docs](p/oebb/readme.md) | [example code](p/oebb/example.js) | [src](p/oebb/index.js)
 [Nahverkehr Sachsen-Anhalt (NASA)](https://de.wikipedia.org/wiki/Nahverkehrsservice_Sachsen-Anhalt)/[INSA](https://insa.de) | – | [docs](p/insa/readme.md) | [example code](p/insa/example.js) | [src](p/insa/index.js)
+[Nahverkehrsverbund Schleswig-Holstein (NAH.SH)](https://de.wikipedia.org/wiki/Nahverkehrsverbund_Schleswig-Holstein) | [`nahsh-hafas`](https://github.com/juliuste/nahsh-hafas) | [docs](p/nahsh/readme.md) | [example code](p/nahsh/example.js) | [src](p/nahsh/index.js)
 
 [![npm version](https://img.shields.io/npm/v/hafas-client.svg)](https://www.npmjs.com/package/hafas-client)
 [![build status](https://img.shields.io/travis/derhuerst/hafas-client.svg)](https://travis-ci.org/derhuerst/hafas-client)
@@ -32,13 +33,7 @@ npm install hafas-client
 
 ## API
 
-- [`journeys(from, to, [opt])`](docs/journeys.md) – get journeys between locations
+[API documentation](docs/readme.md)
-- [`journeyLeg(ref, lineName, [opt])`](docs/journey-leg.md) – get details for a leg of a journey
-- [`departures(station, [opt])`](docs/departures.md) – query the next departures at a station
-- [`locations(query, [opt])`](docs/locations.md) – find stations, POIs and addresses
-- [`location(id)`](docs/location.md) – get details about a location
-- [`nearby(location, [opt])`](docs/nearby.md) – show stations & POIs around
-- [`radar({north, west, south, east}, [opt])`](docs/radar.md) – find all vehicles currently in a certain area
 
 
 ## Usage

test/index.js

@@ -4,4 +4,5 @@ require('./db')
 require('./vbb')
 require('./oebb')
 require('./insa')
+require('./nahsh')
 require('./throttle')

test/nahsh.js

@@ -0,0 +1,452 @@
+'use strict'
+
+// todo
+// const getStations = require('db-stations').full
+const tapePromise = require('tape-promise').default
+const tape = require('tape')
+const isRoughlyEqual = require('is-roughly-equal')
+const validateFptf = require('validate-fptf')
+
+const validateLineWithoutMode = require('./validate-line-without-mode')
+
+const co = require('./co')
+const createClient = require('..')
+const nahshProfile = require('../p/nahsh')
+const {allProducts} = require('../p/nahsh/products')
+const {
+	assertValidStation,
+	assertValidPoi,
+	assertValidAddress,
+	assertValidLocation,
+	assertValidStopover,
+	hour, createWhen, assertValidWhen
+} = require('./util.js')
+
+const when = createWhen('Europe/Berlin', 'de-DE')
+
+const assertValidStationProducts = (t, p) => {
+	t.ok(p)
+	t.equal(typeof p.nationalExp, 'boolean')
+	t.equal(typeof p.national, 'boolean')
+	t.equal(typeof p.interregional, 'boolean')
+	t.equal(typeof p.regional, 'boolean')
+	t.equal(typeof p.suburban, 'boolean')
+	t.equal(typeof p.bus, 'boolean')
+	t.equal(typeof p.ferry, 'boolean')
+	t.equal(typeof p.subway, 'boolean')
+	t.equal(typeof p.tram, 'boolean')
+	t.equal(typeof p.onCall, 'boolean')
+}
+
+const isKielHbf = (s) => {
+	return s.type === 'station' &&
+	(s.id === '8000199') &&
+	s.name === 'Kiel Hbf' &&
+	s.location &&
+	isRoughlyEqual(s.location.latitude, 54.314982, .0005) &&
+	isRoughlyEqual(s.location.longitude, 10.131976, .0005)
+}
+
+const assertIsKielHbf = (t, s) => {
+	t.equal(s.type, 'station')
+	t.ok(s.id === '8000199', 'id should be 8000199')
+	t.equal(s.name, 'Kiel Hbf')
+	t.ok(s.location)
+	t.ok(isRoughlyEqual(s.location.latitude, 54.314982, .0005))
+	t.ok(isRoughlyEqual(s.location.longitude, 10.131976, .0005))
+}
+
+// todo: DRY with assertValidStationProducts
+// todo: DRY with other tests
+const assertValidProducts = (t, p) => {
+	for (let product of allProducts) {
+		product = product.product // wat
+		t.equal(typeof p[product], 'boolean', 'product ' + p + ' must be a boolean')
+	}
+}
+
+const assertValidPrice = (t, p) => {
+	t.ok(p)
+	if (p.amount !== null) {
+		t.equal(typeof p.amount, 'number')
+		t.ok(p.amount > 0)
+	}
+	if (p.hint !== null) {
+		t.equal(typeof p.hint, 'string')
+		t.ok(p.hint)
+	}
+}
+
+const assertValidLine = (t, l) => { // with optional mode
+	const validators = Object.assign({}, validateFptf.defaultValidators, {
+		line: validateLineWithoutMode
+	})
+	const recurse = validateFptf.createRecurse(validators)
+	try {
+		recurse(['line'], l, 'line')
+	} catch (err) {
+		t.ifError(err)
+	}
+}
+
+const test = tapePromise(tape)
+const client = createClient(nahshProfile)
+
+const kielHbf = '8000199'
+const flensburg = '8000103'
+const luebeckHbf = '8000237'
+const husum = '8000181'
+const schleswig = '8005362'
+
+test('Kiel Hbf to Flensburg', co(function* (t) {
+	const journeys = yield client.journeys(kielHbf, flensburg, {
+		when, passedStations: true
+	})
+
+	t.ok(Array.isArray(journeys))
+	t.ok(journeys.length > 0, 'no journeys')
+	for (let journey of journeys) {
+		t.equal(journey.type, 'journey')
+
+		assertValidStation(t, journey.origin)
+		assertValidStationProducts(t, journey.origin.products)
+		// todo
+		// if (!(yield findStation(journey.origin.id))) {
+		// 	console.error('unknown station', journey.origin.id, journey.origin.name)
+		// }
+		if (journey.origin.products) {
+			assertValidProducts(t, journey.origin.products)
+		}
+		assertValidWhen(t, journey.departure, when)
+
+		assertValidStation(t, journey.destination)
+		assertValidStationProducts(t, journey.origin.products)
+		// todo
+		// if (!(yield findStation(journey.origin.id))) {
+		// 	console.error('unknown station', journey.destination.id, journey.destination.name)
+		// }
+		if (journey.destination.products) {
+			assertValidProducts(t, journey.destination.products)
+		}
+		assertValidWhen(t, journey.arrival, when)
+
+		t.ok(Array.isArray(journey.legs))
+		t.ok(journey.legs.length > 0, 'no legs')
+		const leg = journey.legs[0]
+
+		assertValidStation(t, leg.origin)
+		assertValidStationProducts(t, leg.origin.products)
+		// todo
+		// if (!(yield findStation(leg.origin.id))) {
+		// 	console.error('unknown station', leg.origin.id, leg.origin.name)
+		// }
+		assertValidWhen(t, leg.departure, when)
+		t.equal(typeof leg.departurePlatform, 'string')
+
+		assertValidStation(t, leg.destination)
+		assertValidStationProducts(t, leg.origin.products)
+		// todo
+		// if (!(yield findStation(leg.destination.id))) {
+		// 	console.error('unknown station', leg.destination.id, leg.destination.name)
+		// }
+		assertValidWhen(t, leg.arrival, when)
+		t.equal(typeof leg.arrivalPlatform, 'string')
+
+		assertValidLine(t, leg.line)
+
+		t.ok(Array.isArray(leg.passed))
+		for (let stopover of leg.passed) assertValidStopover(t, stopover)
+
+		if (journey.price) assertValidPrice(t, journey.price)
+	}
+
+	t.end()
+}))
+
+test('Kiel Hbf to Husum, Zingel 10', co(function* (t) {
+	const zingel = {
+		type: 'location',
+		latitude: 54.475359,
+		longitude: 9.050798,
+		address: 'Husum, Zingel 10'
+	}
+
+	const journeys = yield client.journeys(kielHbf, zingel, {when})
+
+	t.ok(Array.isArray(journeys))
+	t.ok(journeys.length >= 1, 'no journeys')
+	const journey = journeys[0]
+	const firstLeg = journey.legs[0]
+	const lastLeg = journey.legs[journey.legs.length - 1]
+
+	assertValidStation(t, firstLeg.origin)
+	assertValidStationProducts(t, firstLeg.origin.products)
+	// todo
+	// if (!(yield findStation(leg.origin.id))) {
+	// 	console.error('unknown station', leg.origin.id, leg.origin.name)
+	// }
+	if (firstLeg.origin.products) assertValidProducts(t, firstLeg.origin.products)
+	assertValidWhen(t, firstLeg.departure, when)
+	assertValidWhen(t, firstLeg.arrival, when)
+	assertValidWhen(t, lastLeg.departure, when)
+	assertValidWhen(t, lastLeg.arrival, when)
+
+	const d = lastLeg.destination
+	assertValidAddress(t, d)
+	t.equal(d.address, 'Husum, Zingel 10')
+	t.ok(isRoughlyEqual(.0001, d.latitude, 54.475359))
+	t.ok(isRoughlyEqual(.0001, d.longitude, 9.050798))
+
+	t.end()
+}))
+
+test('Holstentor to Kiel Hbf', co(function* (t) {
+	const holstentor = {
+		type: 'location',
+		latitude: 53.866321,
+		longitude: 10.679976,
+		name: 'Hansestadt Lübeck, Holstentor (Denkmal)',
+		id: '970003547'
+	}
+	const journeys = yield client.journeys(holstentor, kielHbf, {when})
+
+	t.ok(Array.isArray(journeys))
+	t.ok(journeys.length >= 1, 'no journeys')
+	const journey = journeys[0]
+	const firstLeg = journey.legs[0]
+	const lastLeg = journey.legs[journey.legs.length - 1]
+
+	const o = firstLeg.origin
+	assertValidPoi(t, o)
+	t.equal(o.name, 'Hansestadt Lübeck, Holstentor (Denkmal)')
+	t.ok(isRoughlyEqual(.0001, o.latitude, 53.866321))
+	t.ok(isRoughlyEqual(.0001, o.longitude, 10.679976))
+
+	assertValidWhen(t, firstLeg.departure, when)
+	assertValidWhen(t, firstLeg.arrival, when)
+	assertValidWhen(t, lastLeg.departure, when)
+	assertValidWhen(t, lastLeg.arrival, when)
+
+	assertValidStation(t, lastLeg.destination)
+	assertValidStationProducts(t, lastLeg.destination.products)
+	if (lastLeg.destination.products) assertValidProducts(t, lastLeg.destination.products)
+	// todo
+	// if (!(yield findStation(leg.destination.id))) {
+	// 	console.error('unknown station', leg.destination.id, leg.destination.name)
+	// }
+
+	t.end()
+}))
+
+test('Husum to Lübeck Hbf with stopover at Husum', co(function* (t) {
+	const [journey] = yield client.journeys(husum, luebeckHbf, {
+		via: kielHbf,
+		results: 1,
+		when
+	})
+
+	const i1 = journey.legs.findIndex(leg => leg.destination.id === kielHbf)
+	t.ok(i1 >= 0, 'no leg with Kiel Hbf as destination')
+
+	const i2 = journey.legs.findIndex(leg => leg.origin.id === kielHbf)
+	t.ok(i2 >= 0, 'no leg with Kiel Hbf as origin')
+	t.ok(i2 > i1, 'leg with Kiel Hbf as origin must be after leg to it')
+
+	t.end()
+}))
+
+test('earlier/later journeys, Kiel Hbf -> Flensburg', co(function* (t) {
+	const model = yield client.journeys(kielHbf, flensburg, {
+		results: 3, when
+	})
+
+	t.equal(typeof model.earlierRef, 'string')
+	t.ok(model.earlierRef)
+	t.equal(typeof model.laterRef, 'string')
+	t.ok(model.laterRef)
+
+	// when and earlierThan/laterThan should be mutually exclusive
+	t.throws(() => {
+		client.journeys(kielHbf, flensburg, {
+			when, earlierThan: model.earlierRef
+		})
+	})
+	t.throws(() => {
+		client.journeys(kielHbf, flensburg, {
+			when, laterThan: model.laterRef
+		})
+	})
+
+	let earliestDep = Infinity, latestDep = -Infinity
+	for (let j of model) {
+		const dep = +new Date(j.departure)
+		if (dep < earliestDep) earliestDep = dep
+		else if (dep > latestDep) latestDep = dep
+	}
+
+	const earlier = yield client.journeys(kielHbf, flensburg, {
+		results: 3,
+		// todo: single journey ref?
+		earlierThan: model.earlierRef
+	})
+	for (let j of earlier) {
+		t.ok(new Date(j.departure) < earliestDep)
+	}
+
+	const later = yield client.journeys(kielHbf, flensburg, {
+		results: 3,
+		// todo: single journey ref?
+		laterThan: model.laterRef
+	})
+	for (let j of later) {
+		t.ok(new Date(j.departure) > latestDep)
+	}
+
+	t.end()
+}))
+
+test('leg details for Flensburg to Husum', co(function* (t) {
+	const journeys = yield client.journeys(flensburg, husum, {
+		results: 1, when
+	})
+
+	const p = journeys[0].legs[0]
+	t.ok(p.id, 'precondition failed')
+	t.ok(p.line.name, 'precondition failed')
+	const leg = yield client.journeyLeg(p.id, p.line.name, {when})
+
+	t.equal(typeof leg.id, 'string')
+	t.ok(leg.id)
+
+	assertValidLine(t, leg.line)
+
+	t.equal(typeof leg.direction, 'string')
+	t.ok(leg.direction)
+
+	t.ok(Array.isArray(leg.passed))
+	for (let passed of leg.passed) assertValidStopover(t, passed)
+
+	t.end()
+}))
+
+test('departures at Kiel Hbf', co(function* (t) {
+	const deps = yield client.departures(kielHbf, {
+		duration: 30, when
+	})
+
+	t.ok(Array.isArray(deps))
+	for (let dep of deps) {
+		assertValidStation(t, dep.station)
+		assertValidStationProducts(t, dep.station.products)
+		// todo
+		// if (!(yield findStation(dep.station.id))) {
+		// 	console.error('unknown station', dep.station.id, dep.station.name)
+		// }
+		if (dep.station.products) assertValidProducts(t, dep.station.products)
+		assertValidWhen(t, dep.when, when)
+	}
+
+	t.end()
+}))
+
+test('nearby Kiel Hbf', co(function* (t) {
+	const kielHbfPosition = {
+		type: 'location',
+		latitude: 54.314982,
+		longitude: 10.131976
+	}
+	const nearby = yield client.nearby(kielHbfPosition, {
+		results: 2, distance: 400
+	})
+
+	t.ok(Array.isArray(nearby))
+	t.equal(nearby.length, 2)
+
+	assertIsKielHbf(t, nearby[0])
+	t.ok(nearby[0].distance >= 0)
+	t.ok(nearby[0].distance <= 100)
+
+	for (let n of nearby) {
+		if (n.type === 'station') assertValidStation(t, n)
+		else assertValidLocation(t, n)
+	}
+
+	t.end()
+}))
+
+test('locations named Kiel', co(function* (t) {
+	const locations = yield client.locations('Kiel', {
+		results: 10
+	})
+
+	t.ok(Array.isArray(locations))
+	t.ok(locations.length > 0)
+	t.ok(locations.length <= 10)
+
+	for (let l of locations) {
+		if (l.type === 'station') assertValidStation(t, l)
+		else assertValidLocation(t, l)
+	}
+	t.ok(locations.some(isKielHbf))
+
+	t.end()
+}))
+
+test('location', co(function* (t) {
+	const loc = yield client.location(schleswig)
+
+	assertValidStation(t, loc)
+	t.equal(loc.id, schleswig)
+
+	t.end()
+}))
+
+// todo: see #34
+test.skip('radar Kiel', co(function* (t) {
+	const vehicles = yield client.radar(54.4, 10.0, 54.2, 10.2, {
+		duration: 5 * 60, when
+	})
+
+	t.ok(Array.isArray(vehicles))
+	t.ok(vehicles.length > 0)
+	for (let v of vehicles) {
+
+		// todo
+		// t.ok(findStation(v.direction))
+		assertValidLine(t, v.line)
+
+		t.equal(typeof v.location.latitude, 'number')
+		t.ok(v.location.latitude <= 57, 'vehicle is too far away')
+		t.ok(v.location.latitude >= 51, 'vehicle is too far away')
+		t.equal(typeof v.location.longitude, 'number')
+		t.ok(v.location.longitude >= 7, 'vehicle is too far away')
+		t.ok(v.location.longitude <= 13, 'vehicle is too far away')
+
+		t.ok(Array.isArray(v.nextStops))
+		for (let st of v.nextStops) {
+			assertValidStopover(t, st, true)
+
+			if (st.arrival) {
+				t.equal(typeof st.arrival, 'string')
+				const arr = +new Date(st.arrival)
+				// note that this can be an ICE train
+				t.ok(isRoughlyEqual(14 * hour, +when, arr))
+			}
+			if (st.departure) {
+				t.equal(typeof st.departure, 'string')
+				const dep = +new Date(st.departure)
+				t.ok(isRoughlyEqual(14 * hour, +when, dep))
+			}
+		}
+
+		t.ok(Array.isArray(v.frames))
+		for (let f of v.frames) {
+			assertValidStation(t, f.origin, true)
+			assertValidStationProducts(t, f.origin.products)
+			assertValidStation(t, f.destination, true)
+			assertValidStationProducts(t, f.destination.products)
+			t.equal(typeof f.t, 'number')
+		}
+	}
+	t.end()
+}))