Hong Kong Address Parser

An open source project for turning unstructured HK address strings into address components and coordinates, with validation.

What is the origin of this Project? 

As a journalist, sometimes we encounter a bunch of unstructured addresses, and we have to make those structural to extract address components (eg. district/road) for further analysis, geocode for visualization. However the whole cleaning process is such a mess.

Currently we can lookup addresses by the following tools: The OGCIO Address Lookup Service (ALS) and Google Geocoding API (GAPI), however there are some drawbacks:

1. Garbage in, Garbage out: ALS always returns suggested address result, even you just type an ’A’ (try this and see what happen!)
2. Ignore floor and unit: ALS doesn’t support floor and unit, including floor and unit address may corrupt the result
3. Uncertain reliability: GAPI doesn’t return results if the system returns multiple matches (Try searching 將軍澳富麗花園 in Google map and see what happen!)
4. No validation: We never know what we did wrong when the result is wrong

What social problem are you trying to solve?

• Build a rigid geocoder: It saves a lot of tedious work (and brain cells) of verifying geocoding results manually, as a journalist rather go without than have something shoddy, they can’t bear any single mistake.
• Help data analysis: It can make any free-form address structural to facilitate further analysis. Imagine if you enter 「逸東酒店」 or ’Eaton Hotel’, you can get the different levels of address data (eg. 九龍、油麻地、彌敦道) immediately, with bulk processing quick analysis in different district level would be achieved.

How did we begin from scratch?

Making use of the libpostal library, we are going to normalize the street addresses using statistical NLP (or just using fuzzy matching for other cases). For data source, the ’correct addressing’ data from the Hong Kong Post and the street name from data.gov.hk would be used to validate the request address components.

Before parsing any address, our parser should be able to first identify and validate the address components (to check if essential address components are filled), and then pass it to ALS for geocoding and get ’official’ suggested address.

Finally, a web front end similar to usaddress parserator which support bulk parsing and geocoding would be built.

The Challenges we face?

What to do:

• Build a dictionary to deal with abnormal Unicode characters (温 vs 溫) and fullwidth character forms (１vs 1)
• Separate flat and unit number using regex or relevant techniques
• Build a scraper to regularly grasp the address component data (check below)
• Build a database to store the data scrapped
• Integrate the libpostal library with our data
• Set up an API
• Build a web-based front end for dummies
• Develop bulk parsing feature (even we can’t achieve anything above, at least we can build an easy-to-use web end directly using ALS for the dummies)
• Make it bilingual

What resource do you need? 

libpostal - a multilingual street address parsing/normalization library

Address Lookup Service - a web service providing lookup function on Hong Kong address records in both Chinese and English aggregated from various Government Bureaux/Departments

Correct addressing - scraped from Hong Kong Post

Street names - obtained from data.gov.hk

Optional: Database of private buildings in HK (Home Affairs Department), Names of buildings (Rating and Valuation Department)

Who are we? 

Brian Leung, Reporter

brianleung1017@gmail.com

Relevent links:

【開放數據】政府「資料一線通」7成資料屬水份 一圖睇清有啲乜