If you’ve read any book on gathering Twitter data via the API it is highly likely that you will get a fairly standard set of instructions, such as these from Python Social Media Analytics by Chatterjee and Krystyanczuk [PacktPub]:
- Create a Twitter account or use your existing one.
- Go to https://apps.twitter.com/ and log in with your account.
- Click on Create your app and submit your phone number. A valid phone number is required for the verification process. You can use your mobile phone number for one account only….
That’s how I learned it from books, and that’s how it was taught in our Data Science Course, but it appears that access to that method is now closed.
As computing science faculties around the world start to welcome a new intake of students, they need to face up to a difficult period ahead.
The Party’s Over
Towards the end of July 2018 I was completing my MSc Data Science project, a semester-long piece of work which relied heavily on access to Twitter’s API for its data. When I went to add an application to my Twitter account I got a nasty surprise!
If you visit https://apps.twitter.com/ now, you will find that you cannot create a new App unless you have a full developer account.
“Starting July 24th, 2018, anyone who wants to create a new Twitter app will need to have an approved developer account. You can apply for a developer account at developer.twitter.com. Once your application has been approved, you’ll be able to create new apps on developer.twitter.com.”
It should be noted that pre-existing apps, created under existing accounts using the former method, will still work for now, but unless you are the owner of an approved developer account you will no longer be able to create new apps, and so will not be able to get authorisation tokens to run your new project.
As an experiment, I applied to have my account upgraded to a Developer one at the end of July to see how long it would take. As of today, 27th August, I am still waiting.
Why does it matter?
This change creates real challenges to Computer Science departments in universities and colleges world-wide. If you teach big data, social media analytics, data mining, data science or similar, here are a bunch of questions for you:
- How are your students going to learn how to use Twitter data now?
- How will your R and Python Data Science courses teach how to use data which is no longer readily available to students?
- If you continue to teach this, how long will it take students to obtain Developer accounts?
- How can you guarantee that they will get them?
- And what it they don’t?
And for developers, there are further issues ahead. The number of apps you can register are being reduced and the rate limiting is getting tighter in the next couple of weeks.
I understand why Twitter is doing this, and I respect their attempts to tackle real issues by removing bots, fake accounts etc. but, like with all big decisions, it does appear that there are unexpected consquences.
A final call to action
Who, in the academic community, has faced up to these issues? Is there a back-channel to Twitter to ensure that students can still be taught to use Twitter data responsibly? Who is lobbying Twitter on behalf of the educational sector?
We all need to raise these issues and ensure continuity of data science education.