Released Neubot 0.4.6 http://www.neubot.org/neubot-0.4.6 Version 0.4.6 is the latest generally-available stable release of Neubot, distributed for Ubuntu >= 10.04, MacOSX >= 10.6, Windows >= XP SP3 and FreeBSD as well as in source format (.tar.gz and .zip). The release contains 152 patches. The diffstat says: 112 files changed, with 4618 lines inserted and 1124 lines deleted. As you probably know, between 0.4.5 and 0.4.6 Neubot has been added to Measurement Lab (M-Lab) <http://www.measurementlab.net/>, a distri- buted server platform that hosts tools that, like Neubot, want to measure the quality and neutrality of Internet connections. (More on that here <http://nexa.polito.it/neubot-in-mlab-en>.) So, this release contains a lot of patches that deal with "M-Lab integration" as we will see. Here are the most relevant changes: o Neubot added to M-Lab. With respect to measurement quality, there is a tremendous advantage in using M-Lab. The availability of more than 60 servers around the world allows to redirect users to the nearest server and the thing we are measuring, "speed", is roughly inversely proportional to the distance from the server. Plus, at higher distance there are much more confounding factors, for example because you have to traverse more ISPs. So, it matters A LOT to be as close as possible to the users. o Neubot privacy policy harmonized to M-Lab one, permission to share (publish) the Internet address now REQUIRED for running tests. Now Neubot is hosted in more than 60 M-Lab servers around the world and M-Lab policy requires that all results must be published as open data on the web. In terms of Neubot privacy policy, this means that only users that have provided the permission to share their Internet address can access and use Neubot servers on the M-Lab platform. Given the advantages of using M-Lab, I quickly realized that it has no practical sense to support users that has given just the permission to collect. Because it's in the mutual interest of Neubot and its users to always collect the best data possible. For this reason, the privacy policy has been changed and Neubot now will not even attempt a test unless it has the permission to share the user Internet address. For users that have already installed Neubot and have not provided the permission to share, this practically means that Neubot will do its best to notify you of that and prompt you to change your privacy settings. Since this is a significant change, I've bumped the major version number of the privacy policy, and the current version is 2.0. o Privacy: Renamed `publish` the permission to `share` the Internet address, for the sake of clarity. The name `share` for the permission has been criticized because it does not indicate clearly the purpose of the treatment. With `share` I meant the permission to publish the Internet address on the Web, so that other researchers could reuse it for carrying out their studies. You see? The word `publish` communicates much better what the purpose of the treatment is, hence the change. Thanks to Eleonora Bassi, who pointed out the confusion and prodded me to change the wording. o Support for running tests via web user interface (plus some additional command line goodies). I've added a new component to the Neubot daemon, called the runner, which takes care of running tests. The novelty is that I've also added a local HTTP API, /api/runner, that allows to contact the runner and ask it to run a test. Of course periodic & automatic tests use this new runner, but there is more: - now it is possible to run tests via the web user interface, invoking the /api/runner from the AJAX code; - now tests invoked from the command line use /api/runner to run the test in the context of the local daemon. The latter is good for user isolation (the test is not run by your user but by _neubot unprivileged user) and performances (the runner knows the nearest M-Lab server and avoids overlapping periodic & automatic w/ manual tests). At the same time, there is not a loss in transparency: during the test, the daemon passes a copy of its logs to the command line process, which, in turn, prints them on the standard error and honors the -v flag. Of course, it's still possible to run command line tests in the context of the command line process itself, using runner.enabled, e.g.:: neubot speedtest -D runner.enabled=0 o Add "simple and stupid" MacOSX notifier. As said above, in this 0.4.5 -> 0.4.6 transition we need a mean to notify users that have not provided the permission to share/publish that Neubot does not work anymore with their privacy settings. While we have means to alert users both on Debian/Ubuntu and Windows, 0.4.5 is lacking this capability on the MacOSX. To start with, I've written a "simple and stupid" notification mechanism that runs each time you login into your Mac and, if it finds that you have "bad" privacy permissions, it opens the browsers and points it to the privacy page. In turn from this page you can update your settings. This is quite a ugly solution and I promise I will look into more clean solutions, such as Growl, for this kind and other notifications. (I ACK that his behavior is annoying, especially if you don't want to change your privacy permissions. So, if you want to completely remove Neubot 0.4.6 from your MacOSX you can run the following command:: $ sudo /usr/local/share/neubot/0.004006999/uninstall.sh I probably not advertised that enough, but each version of Neubot on the MacOS ships with its own version of such script.) o Add scripts to deploy Neubot to M-Lab. o More code rationalizations, cleanups and regression tests. Packages and source archives available here: http://www.neubot.org/download ChangeLog here: https://raw.github.com/neubot/neubot/0.4.6/ChangeLog Diff since previous version here: http://releases.neubot.org/_packages/neubot-0.4.6.patch Commit history here: https://github.com/neubot/neubot/commits/0.4.6 SHA256sums here: https://raw.github.com/neubot/neubot-sha256s/master/0.4.6/SHA256 Errata here: http://www.neubot.org/errata#0.4.6 Data here: http://www.neubot.org/data P.S. Just a quick note from the data-release dept.: I'm busy working on a much more scalable file-based data-collection Neubot backend that fits with M-Lab data collection strategy. This new backend is going to replace the current sqlite3-based schema, that has serious problems of scalability. Part of the work is to come out with a script that exports existing data to the new format, such that all Neubot.org data is available in the same format. In the process of writing this new exporter script, I will use the December results as test case, and they will probably be the first piece of data available using the new format. (Under the new format, we have a hierarchy of folders and the leafs are folders where each file contains the results of a test in JSON format.) Thank you, Simone <http://www.neubot.org/people#basso>