class: center, middle, first # Wikipedia: RecentChanges Stephen LaPorte [@sklaporte](https://twitter.com/sklaporte) --- background-image: url(rc.png) --- class: center, middle, background-image: url(irc.gif) --- class: middle # Why look at RecentChanges? - Tools to fight vandalism - Push notifications / interaction - Realtime visualizations - [Archiving](https://github.com/internetarchive/crawling-for-nomore404) --- # How it works - Internally, uses MediaWiki's `RecentChange::save()` hook, *see [RecentChanges.php](https://github.com/wikimedia/mediawiki-core/blob/0d1beb5d1316dd85c1125476fdf886fbd7c2ca0d/includes/changes/RecentChange.php#L231)* - Broadcast by [ircd](https://wikitech.wikimedia.org/wiki/IRCD) on Wikimedia wikis - Server is `irc.wikimedia.org`, channels are `#lang.project` (for 730+ wikis) - Format: `[page_title] [flags] [URL_of_the_revision] \* [user] \* [size_of_edit] [edit_summary]` (or if you are fluent in regex) ```python r'(\[\[(?P<page_title>.*?)\]\])' r' +((?P<flags>[A-Z\!]+) )?' r'(?P<url>\S*)' r' +\* (?P<user>.*?)' r' \* (\((?P<change_size>[\+\-][0-9]+)\))?' r' ?(?P<summary>.+)?') ``` --- class: middle, tools # Server - [github.com/hatnote/wikimon](https://github.com/hatnote/wikimon) (Python/Twisted) - [github.com/edsu/wikichanges](https://github.com/edsu/wikichanges) (NodeJS) --- class: middle # Client ```javascript var connection = new WebSocket('ws://wikimon.hatnote.com:9000'); connection.onmessage = function(data) { console.log(data); }; ``` --- # Examples - [Listen to Wikipedia](http://listen.hatnote.com) : Plays sounds based, adjusting the pitch based on the size of edits. - [wikipulse](http://wikipulse.herokuapp.com/) : Live edits-per-minute on Wikipedia. - [Wikipedia Live Monitor](http://wikipedia-irc.herokuapp.com/) : Watch for short intervals to indicate trending news. - [Recent Changes Map](http://rcmap.hatnote.com) : Uses the public IP address from unregistered contributors to geolocate edits on a map.  --- class: middle # Further reading - Learn more about the live data : http://meta.wikimedia.org/wiki/Research:Data#IRC_Feeds - How else could we deliver live recent changes data? - Periodically polling the API (talk to [@mhashemi](https://twitter.com/mhashemi)) - XMPP? http://www.mediawiki.org/wiki/Extension:XMLRC - Download this presentation : http://github.com/slaporte/rc-talk