Our third day at the Data Lab Innovation Week, kicked off with a very interesting talk on data ethics from Dr. Alastair Morrison from the University of Glasgow’s School of Computing.

We were guided by Snook session leaders in the creation of personas and users needs / stories – and we used this learning to create these artefacts for three individuals: a developer, a planning officer and a citizen.

1000 ways to squeeze a lemon
1000 ways to squeeze a lemon

We followed this with sessions ¬†on generating ideas. This was a very active and fun-filled session. With over 100 people participating a game of find 100 ways to squeeze a lemon turned into “find 1000 ways to squeeze…”




Then we worked through a series of exercises – creating wild, silly, ‘bad’ ideas, and even acting out some of them. I dread seeing the video of our group assassinating one of our group who mimed being a dead animal….


When we returned to our projects we pulled the two teams working on our challenge together. We created a storyboard, drawing on the earlier work.

At this point – 3pm on day three – some of us could resist no longer the need to code and work with data. Some of us started to look at that (as time to create anything meaningful in terms of a data-driven service) is now very limited. Others continued to look at paper prototypes – which of course have enormous value in the process.

The (non-)availability of data

What became apparent was the dire lack of data (from a Scottish Planning perspective) with which we could work.

We’d reached out to various groups and individuals earlier in the week, using social media, email and phone calls. Some of our requests were cascaded to others.

At just after midnight I received an email from one person with a planning background who will be nameless but who works in the Scottish Government’s Digital Planning, Planning and Architecture Division. Their email was helpful in tone and suggested some approaches. It included:

“In terms of the planning part, information about planning applications is available on every planning authority website and you are able to set up a search and also set up email notifications of any new application which meets the search criteria. I appreciate it might better suit your needs, if you could simply find the information for the whole of Scotland on a single site.”

“Applications are plotted on GIS systems when received and checked against numerous constraint layers held by the planning authority. Some of the constraint information is made available on Council websites / as part of the Local Development Plan but this will be variable across authorities.”

My reply, other than the greeting, is copied verbatim below:

I responded

Thanks very much for replying, particularly so late at night. I note your points which are helpful and will share these with the team.

I am attending the Data Lab’s Innovation Week with 115 other Data Science MSc students. There are seven challenges which we are working on as part of the event. We selected the one regarding Planning and Biodiversity.

The challenges are exploratory to some degree, looking what might be possible. Part of this involves finding data which we can re-use and which, in combination, can create products or services which are greater than the sum of their parts.

One of the difficulties we have is the lack of Open Data for planning applications in Scotland. No Scottish LA currently publishes its data as open data (OD). In short, this means it is in a neutral format, is machine-readable, and licensed for reuse). Having a web page or an alert service does not meet this requirement.

This is despite both the G8 Charter on Open Data and the Scottish Government’s 2015 Open Data Strategy (the writing of which I was part of) obliging Scotland to make its data openly available.

We are in the position that we can access the NBN data set of 219,000,000 + sightings of species, but can’t get any current or historic planning data without scraping it programatically from 32 local authorities.

Thus we cannot look at, for example, sightings of spcecies x in an area before and after planning permission was granted for a development. Did it decline as predicted, rise, or stay the same?

Similarly we can’t look cumulatively across Scotland to see if, for example, 32 separate developments each had an impact on species numbers. One local authority grant an application which destroys the habitat for a species. In itself this might not have a huge national impact, but 10, 20 or 32 such developments might have a devastating cumulative impact.

When I was in post, managing the website at Aberdeen City Council we started publishing open data in March 2010. There was no organisation support, and no psuh from Scottish Government. After faltering, and some heavy lobbying, the authority decided last month to reinstate its OD programme, which is great.

I also led the Scottish Cities Alliance’s 7-city open data programme until I left last year. This should have a positive impact on the Scottish OD ecosystem, but not enough (being only 7 of 32 councils) and not fast enough.

In Aberdeen we also automated uploads of planning notices to the Tell Me Scotland portal shortly about 6 or 7 years ago. Things are actually now worse than they were five years ago. Planning systems changed and data uploads were broken and not restored. Only 67% of local authorities now have planning notices on the portal – and these are closed data.

Similarly, each LA is obliged to upload its planning notice data to the Scottish Government’s Spatial Hub. This is data in a standard, well-structured format. However, once it is there it is closed and no one can access it, reuse it or create new things with it.

Despite the need to innovate and be more efficient in the public sector, it appears that organisations still don’t understand how OD can contribute to that. A study in 2013 by the UK Transparency Board established that Transport for London’s OD programme (which fuelled an industry of apps and developers) had a ROI to London on 58:1 (yes, that is fifty eight).

While Scotland ignores its obligations to provide open data, and shelters behind issues of legal compliance – such as Ordnance Survey restrictive conditions, many English local authorities publish theirs, and even provide developers with API access to the data.

This stance may not be possible much longer. The Open Data institute is working to develop what the new UK Geospatial Commission needs to be and what it needs to tackle (e.g https://theodi.org/article/how-can-we-support-the-publication-and-use-of-more-open-geospatial-data/ ). As part of the initial process, workshops took place in Aberdeen, Cardiff and London. I ran the Aberdeen one with my colleagues at ODI Aberdeen. One of the strong themes that emerged is the need for a much more permissive licensing of OS data – seeing it as a potential generator of innovation, and thus economic activity, rather than a cash-cow for selling licences to re-use the data, which prices small start-ups out of using it.

Since we delivered the Scottish Government’s 2015 OD strategy very little open data, in my view, has been produced across all of Scottish Government and local government. There is still a limited understanding of what open data is and what the socio-economic benefits are of OD.

It would be great if your programme recognised this and found a way to guide, and push, the planning system in Scotland in the right direction!

If you have any questions please do not hesitate to get in touch.

Best wishes


Data Lab Innovation Week – Day Three
Tagged on: