{
 "metadata": {
  "name": "Part 2 - Direct Interface"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# The direct interface for remote computation with IPython"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "A Client is the low-level object which manages your connection to the various Schedulers and the Hub.\n",
      "Everything you do passes through one of these objects, either indirectly or directly."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import os,sys,time\n",
      "import numpy\n",
      "import numpy as np\n",
      "\n",
      "from IPython.core.display import display, Math\n",
      "\n",
      "from IPython import parallel\n",
      "rc = parallel.Client()\n",
      "rc.block = True # let's start synchronous"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 1
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "It has an `ids` property, which is always an up-to-date list of the integer engine IDs currently available."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "rc.ids"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 2,
       "text": [
        "[0, 1, 2, 3]"
       ]
      }
     ],
     "prompt_number": 2
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The most basic function of the Client is to create the View objects,\n",
      "which are the interfaces for actual communication with the engines.\n",
      "\n",
      "There are two basic models for working with engines: Direct and LoadBalanced.\n",
      "Let's start with the DirectView.\n",
      "\n",
      "The simplest case for remote execution, a DirectView of one engine, and another of all engines:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# index-access of a client gives us a DirectView\n",
      "e0 = rc[0]\n",
      "eall = rc[:]\n",
      "even = rc[::2]\n",
      "odd = rc[1::2]\n",
      "\n",
      "# this is the one we are going to use the most:\n",
      "dview = eall\n",
      "map(display, [e0, eall, even, odd]);"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "display_data",
       "text": [
        "<DirectView 0>"
       ]
      },
      {
       "output_type": "display_data",
       "text": [
        "<DirectView [0, 1, 2, 3]>"
       ]
      },
      {
       "output_type": "display_data",
       "text": [
        "<DirectView [0, 2]>"
       ]
      },
      {
       "output_type": "display_data",
       "text": [
        "<DirectView [1, 3]>"
       ]
      }
     ],
     "prompt_number": 3
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now, the only difference from single-engine remote execution is that the code we run happens on all of the engines of a given view:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "for view in (e0, eall, even, odd):\n",
      "    print view, view.apply_sync(os.getpid)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "<DirectView 0> 2898\n",
        "<DirectView [0, 1, 2, 3]> [2898, 2899, 2900, 2901]\n",
        "<DirectView [0, 2]> "
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[2898, 2900]\n",
        "<DirectView [1, 3]> [2899, 2901]\n"
       ]
      }
     ],
     "prompt_number": 4
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "So how do we use these Views to run code?\n",
      "\n",
      "It's all about:\n",
      "\n",
      "    view.apply(f, *args, **kwargs)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We want the interface for remote and parallel execution to be as natural as possible.\n",
      "And what's the most natural unit of execution?  Code!  Simply define a function,\n",
      "just as you would use locally, and instead of calling it, pass it to `view.apply()`,\n",
      "with the remaining arguments just as you would have passed them to the function."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def get_norms(A, levels=[2]):\n",
      "    \"\"\"get all the requested norms for an array\"\"\"\n",
      "    norms = []\n",
      "    for level in levels:\n",
      "        norms.append(numpy.linalg.norm(A, level))\n",
      "    return norms\n",
      "\n",
      "A = numpy.linspace(0, numpy.pi, 1024)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 5
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "get_norms( A, levels=[1,2,3,numpy.inf])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 6,
       "text": [
        "[1608.4954386379741,\n",
        " 58.055762082012201,\n",
        " 19.954367610618046,\n",
        " 3.1415926535897931]"
       ]
      }
     ],
     "prompt_number": 6
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "To call this remotely, simply replace '`get_norms(`' with '`view.apply(get_norms,`':"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "basic = In[6]\n",
      "remote = basic.replace('get_norms(', 'e0.apply(get_norms,')\n",
      "print ' ' * (len(remote) - len(basic)) + basic\n",
      "print remote\n",
      "ip = get_ipython()\n",
      "ip.set_next_input(remote)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "         get_norms( A, levels=[1,2,3,numpy.inf])\n",
        "e0.apply(get_norms, A, levels=[1,2,3,numpy.inf])\n"
       ]
      }
     ],
     "prompt_number": 7
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "e0.apply(get_norms, A, levels=[1,2,3,numpy.inf])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "ename": "RemoteError",
       "evalue": "NameError(global name 'numpy' is not defined)",
       "output_type": "pyerr",
       "traceback": [
        "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
        "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)\u001b[0;32m<string>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m",
        "\u001b[0;32m<ipython-input-5-c22516ec5624>\u001b[0m in \u001b[0;36mget_norms\u001b[0;34m(A, levels)\u001b[0m",
        "\u001b[0;31mNameError\u001b[0m: global name 'numpy' is not defined"
       ]
      }
     ],
     "prompt_number": 8
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This replacement is generally true for turning (almost) any local function call into a remote one.\n",
      "\n",
      "Note that this will probably raise a NameError on numpy. The simplest way to import numpy is to do:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "e0.execute(\"import numpy\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 9,
       "text": [
        "<AsyncResult: finished>"
       ]
      }
     ],
     "prompt_number": 9
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Or you can use the `%px` magic to import numpy on all engines:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%px import numpy"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 10
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Parallel magics will be discussed in more detail a little later."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Functions don\u2019t have to be interactively defined, you can call module functions remotely as well:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "e0.apply(numpy.linalg.norm, A, 2)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 11,
       "text": [
        "58.055762082012201"
       ]
      }
     ],
     "prompt_number": 11
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "e0.apply(os.getpid)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 12,
       "text": [
        "2898"
       ]
      }
     ],
     "prompt_number": 12
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "In Parallel"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can do the same thing with the DirectView of all engines.\n",
      "\n",
      "The DirectView can be readily understood as an Engine Multiplexer -\n",
      "it does the same thing on all of its engines.\n",
      "\n",
      "The only difference between running code on a single remote engine\n",
      "and running code in parallel is how many engines the DirectView is\n",
      "instructed to use."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "dview.apply(numpy.linalg.norm, A, 2)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 13,
       "text": [
        "[58.055762082012201,\n",
        " 58.055762082012201,\n",
        " 58.055762082012201,\n",
        " 58.055762082012201]"
       ]
      }
     ],
     "prompt_number": 13
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "dview.apply(os.getpid)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 14,
       "text": [
        "[2898, 2899, 2900, 2901]"
       ]
      }
     ],
     "prompt_number": 14
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## execute and run\n",
      "\n",
      "You can also run files or strings with `run` and `execute`\n",
      "respectively.\n",
      "\n",
      "For instance, I have a script `myscript.py` that defines a function\n",
      "`mysquare`:\n",
      "\n",
      "    def mysquare(x):\n",
      "        return x*x\n",
      "\n",
      "I can run that remotely, just like I can locally with `%run`, and then I\n",
      "will have `mysquare()`, and any imports and globals from the script in the\n",
      "engine's namespace:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%pycat myscript.py"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 15
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "e0.run(\"myscript.py\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 16,
       "text": [
        "<AsyncResult: finished>"
       ]
      }
     ],
     "prompt_number": 16
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "e0.execute(\"b=mysquare(a)\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 17,
       "text": [
        "<AsyncResult: finished>"
       ]
      }
     ],
     "prompt_number": 17
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "e0['a']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 18,
       "text": [
        "5"
       ]
      }
     ],
     "prompt_number": 18
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "e0['b']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 19,
       "text": [
        "25"
       ]
      }
     ],
     "prompt_number": 19
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The `%px` and `%%px` magics correspond directly to a View.execute call."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%px import os\n",
      "%px pid = os.getpid()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 20
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "dview['pid']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 21,
       "text": [
        "[2898, 2899, 2900, 2901]"
       ]
      }
     ],
     "prompt_number": 21
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Detail: Working with the engine namespace"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The namespace on the engine is accessible to your functions as `globals`.\n",
      "So if you want to work with values that persist in the engine namespace,\n",
      "you just use global variables."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def inc_a(increment):\n",
      "    global a\n",
      "    a += increment\n",
      "\n",
      "print \"   %2i\" % e0['a']\n",
      "for i in range(3):\n",
      "    e0.apply(inc_a, 5)\n",
      "    print \" +  5\"\n",
      "    print \" = %2i\" % e0['a']\n",
      "    print\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "    5\n",
        " +  5\n",
        " = 10\n",
        "\n",
        " +  5\n",
        " = 15\n",
        "\n",
        " +  5\n",
        " = 20\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 22
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "And just like the rest of Python, you don\u2019t have to specify global variables if you aren\u2019t assigning to them:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def mul_by_a(b):\n",
      "    return a*b\n",
      "\n",
      "e0.apply(mul_by_a, 10)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 23,
       "text": [
        "200"
       ]
      }
     ],
     "prompt_number": 23
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "e0['a'] = 12\n",
      "e0.apply(mul_by_a, 10)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 24,
       "text": [
        "120"
       ]
      }
     ],
     "prompt_number": 24
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "If you want to do multiple actions on data, you obviously don\u2019t want to send it every time.\n",
      "For this, we have a `Reference` class.\n",
      "A Reference is just a wrapper for an identifier that gets unserialized by pulling the corresponding object out of the engine namespace."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def is_it_a(b):\n",
      "    return a is b\n",
      "\n",
      "e0.apply(is_it_a, 5)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 25,
       "text": [
        "False"
       ]
      }
     ],
     "prompt_number": 25
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "e0.apply(is_it_a, parallel.Reference('a'))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 26,
       "text": [
        "True"
       ]
      }
     ],
     "prompt_number": 26
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Which essentially ran the following code on the engine:\n",
      "\n",
      "    _ref_uuid_a = a\n",
      "    is_it_a(_ref_uuid_a)"
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Moving data around"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In addition to calling functions and executing code on engines,\n",
      "you can transfer Python objects to and from your IPython session and the engines.\n",
      "In IPython, these operations are called `push` (sending an object to the engines)\n",
      "and `pull` (getting an object from the engines).\n",
      "\n",
      "push takes a dictionary, used to update the remote namespace:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "e0.push(dict(a=1.03234,b=3453))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 27
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "pull takes one or more keys:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "e0.pull('a')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 28,
       "text": [
        "1.03234"
       ]
      }
     ],
     "prompt_number": 28
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "e0.pull(('b','a'))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 29,
       "text": [
        "[3453, 1.03234]"
       ]
      }
     ],
     "prompt_number": 29
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Push to multiple engines will push the same data to all engines."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "dview.push(dict(a=1.03234,b=3453,c=123));"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 30
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "And pull returns a list of the same length as the number of engines:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "dview.pull('pid')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 31,
       "text": [
        "[2898, 2899, 2900, 2901]"
       ]
      }
     ],
     "prompt_number": 31
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "dview.pull(('c','pid'))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 32,
       "text": [
        "[[123, 2898], [123, 2899], [123, 2900], [123, 2901]]"
       ]
      }
     ],
     "prompt_number": 32
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Dictionary interface"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "treating a DirectView like a dictionary results in push/pull operations:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "e0['a'] = range(5)\n",
      "e0.execute('b = a[::-1]')\n",
      "e0['b']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 33,
       "text": [
        "[4, 3, 2, 1, 0]"
       ]
      }
     ],
     "prompt_number": 33
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%px pid = os.getpid()\n",
      "dview['pid']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 34,
       "text": [
        "[2898, 2899, 2900, 2901]"
       ]
      }
     ],
     "prompt_number": 34
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`get()` and `update()` work as well."
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "scatter/gather"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "push/pull move whole objects around,\n",
      "but sometimes you want to partition objects across the engines.\n",
      "\n",
      "DirectViews have `scatter()` and `gather()` methods, to help with this.\n",
      "Pass any container or numpy array, and IPython will partition the object onto the engines wih `scatter`,\n",
      "or reconstruct the full object in the Client with `gather()`."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "dview.scatter('a', range(16))\n",
      "dview['a']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 35,
       "text": [
        "[[0, 1, 2, 3], [4, 5, 6, 7], [8, 9, 10, 11], [12, 13, 14, 15]]"
       ]
      }
     ],
     "prompt_number": 35
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "dview.gather('a')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 36,
       "text": [
        "[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]"
       ]
      }
     ],
     "prompt_number": 36
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "dview.execute(\"asum = sum(a)\")\n",
      "dview.gather('asum')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 37,
       "text": [
        "[6, 22, 38, 54]"
       ]
      }
     ],
     "prompt_number": 37
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can pass a 'flatten' keyword,\n",
      "to instruct engines that will only get one item of the list to\n",
      "get the actual item, rather than a one-element sublist."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "dview.scatter('rank', rc.ids)\n",
      "dview['rank']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 38,
       "text": [
        "[[0], [1], [2], [3]]"
       ]
      }
     ],
     "prompt_number": 38
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "dview.scatter('rank', rc.ids, flatten=True)\n",
      "dview['rank']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 39,
       "text": [
        "[0, 1, 2, 3]"
       ]
      }
     ],
     "prompt_number": 39
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This particular call is very common when you want MPI-style switching for the behavior of code depending on then engine."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "A simple scatter/gather of numpy arrays:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "A = np.random.randint(1,10,(16,4))\n",
      "B = np.random.randint(1,10,(4,16))\n",
      "display(Math('A ='))\n",
      "display(A)\n",
      "display(Math('B ='))\n",
      "display(B)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "latex": [
        "$$A =$$"
       ],
       "output_type": "display_data",
       "text": [
        "<IPython.core.display.Math at 0x1080a30d0>"
       ]
      },
      {
       "output_type": "display_data",
       "text": [
        "array([[2, 3, 1, 7],\n",
        "       [2, 8, 4, 6],\n",
        "       [9, 9, 2, 9],\n",
        "       [8, 7, 2, 2],\n",
        "       [1, 2, 6, 6],\n",
        "       [1, 1, 3, 4],\n",
        "       [7, 9, 8, 7],\n",
        "       [5, 5, 4, 9],\n",
        "       [4, 4, 8, 5],\n",
        "       [5, 3, 3, 3],\n",
        "       [2, 4, 9, 2],\n",
        "       [1, 2, 3, 3],\n",
        "       [1, 6, 8, 7],\n",
        "       [7, 8, 5, 3],\n",
        "       [5, 9, 9, 7],\n",
        "       [1, 2, 4, 8]])"
       ]
      },
      {
       "latex": [
        "$$B =$$"
       ],
       "output_type": "display_data",
       "text": [
        "<IPython.core.display.Math at 0x1087bcd90>"
       ]
      },
      {
       "output_type": "display_data",
       "text": [
        "array([[6, 9, 6, 6, 5, 1, 1, 3, 7, 3, 4, 8, 5, 6, 7, 9],\n",
        "       [2, 9, 6, 3, 7, 1, 1, 7, 2, 6, 1, 5, 5, 6, 3, 5],\n",
        "       [1, 1, 4, 3, 3, 8, 4, 9, 7, 2, 8, 2, 3, 4, 2, 4],\n",
        "       [4, 1, 8, 2, 2, 7, 6, 8, 4, 4, 5, 2, 5, 2, 7, 6]])"
       ]
      }
     ],
     "prompt_number": 40
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "dview.scatter('A', A)\n",
      "dview.scatter('B', B)\n",
      "display(Math('A_0 ='))\n",
      "display(e0['A'])\n",
      "display(Math('B_0 ='))\n",
      "display(e0['B'])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "latex": [
        "$$A_0 =$$"
       ],
       "output_type": "display_data",
       "text": [
        "<IPython.core.display.Math at 0x10ba82590>"
       ]
      },
      {
       "output_type": "display_data",
       "text": [
        "array([[9, 4, 2, 3],\n",
        "       [4, 3, 7, 1],\n",
        "       [5, 7, 5, 2],\n",
        "       [8, 4, 5, 7]])"
       ]
      },
      {
       "latex": [
        "$$B_0 =$$"
       ],
       "output_type": "display_data",
       "text": [
        "<IPython.core.display.Math at 0x10ba82590>"
       ]
      },
      {
       "output_type": "display_data",
       "text": [
        "array([[4, 5, 8, 8, 7, 4, 7, 6, 3, 5, 2, 7, 5, 9, 3, 7]])"
       ]
      }
     ],
     "prompt_number": 38
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Asynchronous execution"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We have covered the basic methods for running code remotely, but we have been using `block=True`.  We can also do non-blocking execution (this wouldn't be especially useful if we couldn't!)."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "e0.block = dview.block = False"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 41
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In non-blocking mode, `apply` submits the command to be executed and\n",
      "then returns a `AsyncResult` object immediately. The `AsyncResult`\n",
      "object gives you a way of getting a result at a later time through its\n",
      "`get()` method.\n",
      "\n",
      "The AsyncResult object provides a superset of the interface in [`multiprocessing.pool.AsyncResult`](http://docs.python.org/library/multiprocessing#multiprocessing.pool.AsyncResult).\n",
      "See the official Python documentation for more."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def wait(t):\n",
      "     import time\n",
      "     tic = time.time()\n",
      "     time.sleep(t)\n",
      "     return time.time()-tic"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 42
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "ar = e0.apply(wait, 10)\n",
      "ar"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 43,
       "text": [
        "<AsyncResult: wait>"
       ]
      }
     ],
     "prompt_number": 43
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`ar.ready()` tells us if the result is ready"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "ar.ready()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 44,
       "text": [
        "False"
       ]
      }
     ],
     "prompt_number": 44
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`ar.get()` blocks until the result is ready, or a timeout is reached, if one is specified"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%time ar.get(1)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "ename": "TimeoutError",
       "evalue": "Result not ready.",
       "output_type": "pyerr",
       "traceback": [
        "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[0;31mTimeoutError\u001b[0m                              Traceback (most recent call last)",
        "\u001b[0;32m<ipython-input-45-17b98a143e5c>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mget_ipython\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmagic\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34mu'time ar.get(1)'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
        "\u001b[0;32m/Users/bgranger/Documents/Computation/IPython/code/ipython/IPython/core/interactiveshell.pyc\u001b[0m in \u001b[0;36mmagic\u001b[0;34m(self, arg_s)\u001b[0m\n\u001b[1;32m   2159\u001b[0m         \u001b[0mmagic_name\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0m_\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmagic_arg_s\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0marg_s\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpartition\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m' '\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2160\u001b[0m         \u001b[0mmagic_name\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmagic_name\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlstrip\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprefilter\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mESC_MAGIC\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2161\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun_line_magic\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmagic_name\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmagic_arg_s\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2162\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2163\u001b[0m     \u001b[0;31m#-------------------------------------------------------------------------\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
        "\u001b[0;32m/Users/bgranger/Documents/Computation/IPython/code/ipython/IPython/core/interactiveshell.pyc\u001b[0m in \u001b[0;36mrun_line_magic\u001b[0;34m(self, magic_name, line)\u001b[0m\n\u001b[1;32m   2085\u001b[0m                 \u001b[0mkwargs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'local_ns'\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msys\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_getframe\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstack_depth\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mf_locals\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2086\u001b[0m             \u001b[0;32mwith\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbuiltin_trap\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2087\u001b[0;31m                 \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2088\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mresult\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2089\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
        "\u001b[0;32m/Users/bgranger/Documents/Computation/IPython/code/ipython/IPython/core/magics/execution.pyc\u001b[0m in \u001b[0;36mtime\u001b[0;34m(self, parameter_s, local_ns)\u001b[0m\n",
        "\u001b[0;32m/Users/bgranger/Documents/Computation/IPython/code/ipython/IPython/core/magic.pyc\u001b[0m in \u001b[0;36m<lambda>\u001b[0;34m(f, *a, **k)\u001b[0m\n\u001b[1;32m    189\u001b[0m     \u001b[0;31m# but it's overkill for just that one bit of state.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    190\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mmagic_deco\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 191\u001b[0;31m         \u001b[0mcall\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mlambda\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mk\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mk\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    192\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    193\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mcallable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
        "\u001b[0;32m/Users/bgranger/Documents/Computation/IPython/code/ipython/IPython/core/magics/execution.pyc\u001b[0m in \u001b[0;36mtime\u001b[0;34m(self, parameter_s, local_ns)\u001b[0m\n\u001b[1;32m    901\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mmode\u001b[0m\u001b[0;34m==\u001b[0m\u001b[0;34m'eval'\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    902\u001b[0m             \u001b[0mst\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mclock2\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 903\u001b[0;31m             \u001b[0mout\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0meval\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcode\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mglob\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlocal_ns\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    904\u001b[0m             \u001b[0mend\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mclock2\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    905\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
        "\u001b[0;32m<timed eval>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n",
        "\u001b[0;32m/Users/bgranger/Documents/Computation/IPython/code/ipython/IPython/parallel/client/asyncresult.pyc\u001b[0m in \u001b[0;36mget\u001b[0;34m(self, timeout)\u001b[0m\n\u001b[1;32m    126\u001b[0m                 \u001b[0;32mraise\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_exception\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    127\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 128\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0merror\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mTimeoutError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Result not ready.\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    129\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    130\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_check_ready\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
        "\u001b[0;31mTimeoutError\u001b[0m: Result not ready."
       ]
      }
     ],
     "prompt_number": 45
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%time ar.get()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "CPU times: user 0.02 s, sys: 0.01 s, total: 0.03 s\n",
        "Wall time: 0.14 s\n"
       ]
      },
      {
       "output_type": "pyout",
       "prompt_number": 46,
       "text": [
        "10.00037693977356"
       ]
      }
     ],
     "prompt_number": 46
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "For convenience, you can set block for a single call with the extra sync/async methods:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "e0.apply_sync(os.getpid)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 48,
       "text": [
        "2898"
       ]
      }
     ],
     "prompt_number": 48
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "e0.apply_async(os.getpid)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 49,
       "text": [
        "<AsyncResult: getpid>"
       ]
      }
     ],
     "prompt_number": 49
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "_.get()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 50,
       "text": [
        "2898"
       ]
      }
     ],
     "prompt_number": 50
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The same difference between single and multiengine applies here\n",
      "\n",
      "(in fact, blocking execution is actually non-blocking execution with an implied `asyncresult.get()` at the end!)"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "ar = dview.apply_async(os.getpid)\n",
      "ar"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 51,
       "text": [
        "<AsyncResult: getpid>"
       ]
      }
     ],
     "prompt_number": 51
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "ar.get()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 52,
       "text": [
        "[2898, 2899, 2900, 2901]"
       ]
      }
     ],
     "prompt_number": 52
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Map"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "DirectViews have a map method, which behaves just like the builtin map,\n",
      "but computed in parallel."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "dview.block = True\n",
      "\n",
      "serial_result   =       map(lambda x:x**10, range(32))\n",
      "parallel_result = dview.map(lambda x:x**10, range(32))\n",
      "\n",
      "serial_result == parallel_result"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 53,
       "text": [
        "True"
       ]
      }
     ],
     "prompt_number": 53
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`DirectView.map` partitions the sequences onto each engine,\n",
      "and then calls `map` remotely.  The result is always a single\n",
      "IPython task per engine."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def wait_here(t):\n",
      "    import time\n",
      "    print t\n",
      "    time.sleep(t)\n",
      "    return t"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 54
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The AsyncResult object is the principal API for working with parallel results, pending or complete.\n",
      "It has a few helper methods and properties for monitoring your data:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "times = np.linspace(0,1,17)\n",
      "amr = dview.map_async(wait_here, times)\n",
      "# wait for the tasks to finish\n",
      "amr.wait_interactive()\n",
      "# print some timings\n",
      "print amr.serial_time\n",
      "print amr.wall_time"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "   4/4 tasks finished after    3 s"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "done\n",
        "8.523741\n",
        "3.656976\n"
       ]
      }
     ],
     "prompt_number": 55
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# display the output:\n",
      "amr.display_outputs()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[stdout:0] \n",
        "0.0\n",
        "0.0625\n",
        "0.125\n",
        "0.1875\n",
        "0.25\n",
        "[stdout:1] \n",
        "0.3125\n",
        "0.375\n",
        "0.4375\n",
        "0.5\n",
        "[stdout:2] \n",
        "0.5625\n",
        "0.625\n",
        "0.6875\n",
        "0.75\n",
        "[stdout:3] \n",
        "0.8125\n",
        "0.875\n",
        "0.9375\n",
        "1.0\n"
       ]
      }
     ],
     "prompt_number": 56
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Summary"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "* view.apply calls a function remotely\n",
      "* view.execute runs code strings remotely\n",
      "* apply functions access the engine namespace as globals\n",
      "* push/pull and scatter/gather move data around\n",
      "* Single Engine views return single objects, MultiEngine views return lists\n",
      "* AsyncResults wrap results for waiting and metadata.\n"
     ]
    }
   ],
   "metadata": {}
  }
 ]
}