Notes from MPUG, June 2013

The Melbourne Python User Group met this evening.

Apart from the talk on Python one-liners (see my previous post), we covered a range of other topics, including:

The next MPUG meeting will be July 1, the week before PyCon AU, and there is a Melbourne Django Meetup next week.

Posted in Python | Comments Off on Notes from MPUG, June 2013

Notes from MPUG, June 2013: “Python one-liners” talk

Python one-liners

The Python interpreter command line arguments

Three useful command line arguments for Python one-liners:

    graeme@psylocke:~/Projects/Presentations/MPUG_Jun2013$ python -h
    usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ...
    Options and arguments (and corresponding environment variables):
    -c cmd : program passed in as string (terminates option list)
    -m mod : run library module as a script (terminates option list)
    -      : program read from stdin (default; interactive mode if a tty)

Useful programs & tools

Calendar:

    python -m calendar
    python -m calendar 1999

(type python -m calendar -h for more options)

ZIP and GZIP tools (for when you are stuck at a DOS prompt on a Windows box…)

    python -m zipfile -l pcblib.zip
    python -m zipfile -e pcblib.zip
    python -m zipfile -c release.zip *.py

    python -m gzip -d mydata.tar.gz

Pretty JSON printer:

    python -m json.tool < mydata.json

Unicode character printers, by name or symbol:

    python -c "import unicodedata; print unicodedata.name(unichr(0x2249))"
    python -c "print unichr(0x2249)"

A simple webserver:

    python -m SimpleHTTPServer

You can specify the port number if needed and there is also a CGI version:

    python -m CGIHTTPServer

A basic SMTP server:

    python -m smtpd -n -c DebuggingServer localhost:8025

Again, when you are stuck on a Windows box:

    python -m telnetlib  

There are similar command line tools for FTP, POP, SMTP (client as well as server)
and web.

pydoc tool to read & search Python documentation:

    python -m pydoc 
    python -m pydoc -k 

Python debugger:

    python -m pdb 

Interactive profile statistics browser:

    python -m pstats

String processing:

    python -c "import sys; print ''.join(x.capitalize() for x in sys.stdin)"  < names.txt

Numerical processing:

    python -c "import sys, math; print math.fsum(float(x) for x in sys.stdin)" < numbers.dat

(From Andrew Walker) "what is the easiest way to get a URL encoded password?" With a Python one liner!

    python -c 'from getpass import getpass; from urllib import quote; print quote(getpass("Proxy Password: "))'

Clever/quirky one-liners

The Zen of Python:

    python -m this
    python -c "import this"

xkcd 353:

    python -c "import antigravity"

Braces instead of whitespace?

    python -c "from __future__ import braces"

A CS lecture:

    python -c "import heapq; print heapq.__about__"

How does '-m' work?

    if __name__ == '__main__':
        print "do stuff..."

Some others that don't quite fit into this category

A bash function to grep on all Python files in the current directory:

    pygrep () { grep $* *.py; }

Find the source code directory for a Python module (from here )

    cdp () {
        cd "$(python -c "import os.path as _, ${1}; \
        print _.dirname(_.realpath(${1}.__file__[:-1]))"
        )"
    }

Other useful resources

  • The 'e' module

  • A great discussion at StackOverflow with lots more examples

  • pyp - "Pyp is a linux command line text manipulation tool similar to awk or sed, but which uses standard python string and
    list methods as well as custom functions evolved to generate fast results in an intense production environment."

Posted in Python, Talks | Comments Off on Notes from MPUG, June 2013: “Python one-liners” talk

Notes from MPUG, May 2013

The Melbourne Python User Group had a good turn out for the May meeting with about 30 people coming along for a night of talks followed by discussion over pizza and beer.

Talk 1: Ansible

Javier Candeira walked us through the basics of Ansible, a Python-based systems deployment/configuration/management tool that is similar to tools like Chef and Puppet.

It has a number of advantage over similar tools:

  • It is Python-based, but you can write modules in other languages
  • It is serviceless, only requiring SSH, so no extra ports or PKI
  • It uses plain text files, no database
  • Root isn’t require, you can use sudo

Ansible uses Jinja2 for templating and has some neat concepts around modules and “playbooks”, which are collections of “actions”, written in YAML with a natural language interface.

Along the way, Javier also touched on Cobbler (from Redhat), paramiko (SSH in python) and fireball.

If you want more to find out more about Ansible, start with the presentation from Jan-Piet Mens (PDF).

Talk 2: Nose of Yeti

Stephen Moore talked about Nose of Yeti, his rspec-like testing DSL for nose. It uses Python’s file encoders/decoders to modify Python files, translating the DSL into Python code for testing.

Other modules discussed included PyVows (a BDD framework for Python) and fudge (for mocks and stubs).

Talk 3: Deploying Python programs on Debian

Michael Cooper walked us through methods for packaging Python programs for deployment on Debian, covering setuptools, virtualenv, pip, stdeb, dpkg-build and upstart.

I wasn’t aware of stdeb, which builds Debian source packages from Python packages.

The presentation can be found here.

The June meeting

The next MPUG meeting is this coming Monday, 3rd June, 6pm at Inspire9, 1/41 Stewart Street, Richmond.

Posted in Python | Comments Off on Notes from MPUG, May 2013

Notes from MPUG, March 2013

The March Melbourne Python Users Group meeting had three speakers and a whole of interesting chats over pizza and beer afterwards.

Stewart Hines is the author of Picklets. The Picklet Builder app uses Django with a Google AppEngine backend on managing Picklet file distribution via Dropbox and the Dropbox/Python API. Stewart walked us through what’s involved in using Dropbox with Python.

Loki Davison gave an introduction to Boo (aka “Boo! Not Python but almost…”).

Boo:

  • Targets the CLI/.NET platform
  • Has a very Pythonic syntax (many of the examples will work in either language)
  • Has static typing with optional duck typing
  • Syntactic macros
  • Type inference (for implicit type declarations)

One issue is the lack of good libraries, compared with Python’s rich set of standard and third party libraries.

Andrew Walker is working in an environment where the computer systems can be locked down, making it difficult to use standard tools to install Python packages. This led to him exploring “how many ways are there to install a Python package?”.

Some of the packaging options Andrew walked us through were:

  1. Your O/S package manager (eg. apt-get or yum)
  2. easy_install
  3. setuptools and distribute
  4. pip
    • Maximum flexibility
    • Can uninstall packages
    • Can work with multiple source types, including source, tarballs, Github and PyPI
A key recommendation: if you can, use pip + virtualenv.

Finally, some housekeeping: 

  • The PyCon AU 2013 call for proposals is open and closes on April 5.
  • There is no April MPUG meeting. The next meeting will be Monday, 6 May at 6pm at Inspire 9.
Posted in Python | Comments Off on Notes from MPUG, March 2013

Notes from MPUG, December 2012 – part 2, as well as January & February 2013!

Firstly some housekeeping: there’s no MPUG meeting this month. The next meeting will be on March 4, 6pm at Inspire 9.

The February meeting was moved to January 28 to avoid a clash with Tim Berners-Lee visiting town. Unfortunately the new meeting time clashed with LCA and the Australia Day weekend, which meant a number of people, including myself, couldn’t make it.

In lieu of not posting anything about the Jan/Feb meeting, here is the continuation of my notes from the December 2012 meeting.

Ed Schofield from Python Charmers spoke on 10 cool things about Python:

  1. Inline plotting in IPython – in a “notebook” in a web browser or the Qt console
  2. IPython interfaces – R & Cython as well “magic” interfaces 
  3. The collections module in the Python standard library
  4. dict and set comprehensions
  5. %pdb magic in IPython
  6. The Pandas library
    • For data analysis and modelling
    • Came out of the hedge fund industry
    • DataFrame object for manipulating data
    • Excellent support for working with CSV, SQL, Excel files
    • Inbuilt support for pivot tables
  7. The requests library: “HTTP for humans”
  8. WeasyPrint – ACID2-compliant HTML and CSS rendering to PDF
    • Really cool!
    • Uses “print” CSS for formatting
  9. scikit.learn – a general purpose machine learning library for Python 
  10. The Python ecosystem in general

Lars from 99 Designs spoke about Eigenface in analysing logo designs:

  • Eigenface: uses eigenvectors for recognition 
  • Using PCA (Principal Components Analysis) from scikit.learn
  • Some interesting techniques that could be used across a range of image recognition/analysis domains
Posted in Python | Comments Off on Notes from MPUG, December 2012 – part 2, as well as January & February 2013!

fenix revisited

It’s been over four months since I bought my Garmin fenix and wrote up my initial impressions of the watch. Having run, cycled and hiked a lot of kilometres since then, it is worth giving an update on the fenix. Garmin have released another four firmware upgrades and I am now using v3.10 firmware.

Bugs and other problems that Garmin have addressed:

  1. Foot pod support (for running cadence measurements) has been added
  2. Indoor training support has been added
  3. Altitude readings are more reliable, with more options for configuring the combination of barometric and GPS measurements used for tracking altitude
  4. The FIT history is now accessible from the main menu and has a number of bug fixes and improvements with what is tracked in the FIT file

Bugs and problems that still exist (and annoy me)

  1. The GPS can take several minutes to lock onto satellites. I leave it outside to get a lock, go get changed into my running gear and then grab the watch. The lock time is inconsistent and frustratingly slow, especially when compared with other GPS devices that I have.
  2. A number of really useful running features from the FR-60 and other Forerunner watches are still missing, such as Virtual Partner.
  3. The manual is still appalling.
  4. Garmin still don’t have a dedicated forum for the fenix, despite all the requests and forums for most (if not all) of their other sports watches.
  5. There is still no ANT+ or Bluetooth connectivity to a PC. There is Bluetooth connectivity to an iPhone app, and that’s it.
  6. Customising the watch onboard is convenient when you are in the field, compared to the Suunto Ambit which requires a PC for configuration, but the customisation is poorly documented and often counter-intuitive. I find myself wandering up and down menu lists trying to find a particular option. A good example is the group of time/date settings, which are spread across a number of different menus. 
  7. The profile support via the Garmin Connect API is very average. Import a cycling track and it is imported as a running event. Import a running track and it is imported as an “other” event. This means that for every import of tracks, I have to edit the event types to set them correctly. Every import. How hard would it be to fix this, Garmin?

Would I recommend the fenix? If you need a sports watch with long battery life (eg. Audax rides, ultra-marathons, long day walks), then the fenix is a good choice, but I would recommend you also consider the Suunto Ambit, especially now that it has “app” functionality. Otherwise, I would seriously consider one of the alternative sports watches, such as the 910XT, that are on the market, especially if you want training features.

Posted in Cycling, Gear, Running | Comments Off on fenix revisited

Notes from MPUG, December 2012 – part 1

We had about twenty people at the last Melbourne Python User Group meeting for 2012, celebrating with six litres of mulled wine, five dozen fruit mince pies, four speakers speaking and a partridge in a pear tree.

Nathan Faggian and Ed Schofield gave a great overview of image processing in Python, starting with some of the packages available, including scikit-image, which was recommended as very Pythonic. The scikit-image gallery is worth a look.

From there, they covered image segmentation, cellular automata and implementing versions of Conway’s Game of Life with a straight Python version and a version that used numpy for a significant performance improvement. This led to growcut.automata, Nathan’s implementation of the Growcut algorithm for user-interactive image segmentation using cellular automata. The user nominates a few regions within an image as being associated with either the object of interest or the background; Growcut can then segment out the object of interest. The live demonstrations were impressive: segmenting a flower out of a reasonably complex background and then a scanned piece of paper from a photo.

I’d point to Nathan’s code but the repository seems to have 404’ed 🙁

A recurring theme from the December meeting was the use of IPython notebooks for prototyping and demonstrations. If you aren’t familiar with notebooks, some good starting points are:

I’ll write about the other talks from the December meeting in part 2…

Posted in Python | Comments Off on Notes from MPUG, December 2012 – part 1

(No) notes from MPUG, November 2012

I missed the November meeting of the Melbourne Python Users Group as I was holidaying on Flinders Island, so there’s no notes!

Word on the street is that the group got to try out the new meeting room at Inspire 9, that Tennessee spoke on “Hacking science with Ipython Notebook” and Javier covered “Infrastructure as a service with Python, apache-libcloud and Rackspace or AWS”.

The last meeting for the year is this Monday (December 3), 6pm, at Inspire 9 (Level 1, 41 Stewart St, Richmond – just opposite the Richmond train station). There will be talks, pizza and Christmas festivities!

Posted in Python | Comments Off on (No) notes from MPUG, November 2012

Marysville half

173 people died in the Black Saturday bushfires on February 7, 2009. One of the hardest hit towns was Marysville, where 34 people died and only 14 out of about 400 buildings were left standing.

Four years ago a local doctor, Lachlan Fraser, organised the Marysville marathon as part of the healing process, and as a way of bringing people back to Marysville. His initial posting after the bushfire on the Coolrunning website was followed up with plans of organising a running festival for November 2009, which planned for 500+ entrants and ended up with 3000!

Fast forward to 2012, and the fourth Marysville Marathon Festival was held on November 11, with five races: 4, 10, 21.1, 42.2 and 50km.

Start of the Marysville marathon

Pink tractors mark the starting line of the Marysville marathon

Having bludged since the Melbourne Marathon, I entered the half marathon, which was a challenging but beautiful mix of different bush tracks, hills, rivers and only 3.5km of bitumen on the climb up to the Falls.

Altitude profile for the Marysville half marathon

Altitude profile for the Marysville half marathon

Both the township and the surrounding forests have recovered remarkably in four years and it was fantastic to be part of a running event that contributes so much back into to the local community.

Mark it in your calendar for November 2013.

Posted in Running | Comments Off on Marysville half

The Strzelecki Peaks, Flinders Island

The Strzelecki Peaks are the highest peaks on Flinders Island and a great example of the Devonite granite massifs that stretch from Wilson’s Promontory on the mainland across the various Bass Strait islands down to Tasmania.

Walking the Strzelecki Peaks

It is about 3km to the top of the peaks, but there is over 700 metres of climbing!

Cloud rolling in over the Strzelecki Peaks

Cloud rolling in over the Strzelecki Peaks

Trousers Point, Mt Chappell Island, Badger Island

Looking south-west towards Trousers Point, Mt Chappell Island and Badger Island

Looking south along the peaks

Looking south along the peaks

Posted in Hiking, Travel | Comments Off on The Strzelecki Peaks, Flinders Island