Difference between revisions of "IPython"

From Storrs HPC Wiki
Jump to: navigation, search
(Fix typo)
(Introduce the IPython shell)
Line 31: Line 31:
  
 
Learning objectives:
 
Learning objectives:
* Creating macros
+
* Overview of IPython
 +
* Building up a script
 
* Running python programs
 
* Running python programs
* Building up a script
 
 
* Debugging errors
 
* Debugging errors
  
Line 86: Line 86:
 
#> ~/.local/bin/ipython
 
#> ~/.local/bin/ipython
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 +
Run <code>ipython</code>:
 +
<syntaxhighlight lang="bash">
 +
ipython
 +
#> Python 3.6.1 (default, May 13 2017, 20:28:40)
 +
#> Type 'copyright', 'credits' or 'license' for more information
 +
#> IPython 6.0.0 -- An enhanced Interactive Python. Type '?' for help.
 +
#>
 +
#> In [1]:
 +
</syntaxhighlight>
 +
 +
= Overview =
 +
 +
Spent 2 minutes to read the builtin help as suggested by the startup message:
 +
 +
?
 +
 +
Like many IPython commands, the help opens up inside of the <code>less</code> pager.
 +
To navigate,
 +
use your the keyboard commands for <code>less</code> you know and love:
 +
 +
{|class="wikitable"
 +
! Keyboard Shortcut !! Action
 +
|-
 +
| arrow keys || Move one line
 +
|-
 +
| space || Next page
 +
|-
 +
| b || Previous page
 +
|-
 +
| g || Jump to top
 +
|-
 +
| G || Jump to bottom
 +
|-
 +
| /word || Search for "word"
 +
|-
 +
| n || Next search result
 +
|-
 +
| N || Previous search result
 +
|-
 +
| q || Quit
 +
|-
 +
| h || Help
 +
|}
 +
 +
As the help screen explains,
 +
IPython gives us all the power of the shell inside of python.
 +
We can run shell commands inside of Python,
 +
so that we never heave to leave python to run various housekeeping commands.
 +
This is possible using what it calls "magic".
 +
All magic commands begin with a <code>%</code>
 +
including the command to learn about "magic"!
 +
 +
Exit out of the help screen and let's read the short form of magic commands:
 +
 +
%quickref
 +
 +
One can see the equivalent full help with <code>%magic</code>
 +
 +
%magic
 +
 +
As you can see the help is vast.
 +
For now let's look at the few most important commands
 +
by creating our project directories.
 +
 +
<syntaxhighlight lang="bash">
 +
# The [TAB] here means use the TAB key; don't actually type "[TAB]" ^_~
 +
 +
cd /scr[TAB]
 +
#> cd /scratch/
 +
cd /scratch/
 +
pwd
 +
#> '/gpfs/scratchfs1'
 +
ls
 +
#> (Lots of directories!)
 +
mkdir -p abc12345/ipy # Replace with your NetID!
 +
cd abc12345/ipy      # Replace with your NetID!
 +
#> /gpfs/scratchfs1/abc12345/ipy
 +
mkdir data src results
 +
ls
 +
#> data/  results/  src/
 +
!tree $PWD
 +
#> /gpfs/scratchfs1/abc12345/ipy
 +
#> ├── data
 +
#> ├── results
 +
#> └── src
 +
#>
 +
#> 3 directories, 0 files
 +
</syntaxhighlight>
 +
 +
To run any shell command,
 +
we put an exclamation mark <code>!</code> before the command name.
 +
We used the <code>!tree</code> command to recursively print all out directores
 +
in a nice human readable format.
 +
Here <code>$PWD</code> is a shell variable which we are able to access inside of IPython
 +
when running shell commands with <code>!</code>.
 +
Strictly speaking, the <code>$PWD</code> was not necessary,
 +
but it shows the tree with the absolute path
 +
and also demonstrates that shell variables are usable with shell commands.
 +
 +
Now that we have our directory structure
 +
we can move on to the next section of creating our Python script.
  
 
[[Category:Core]]
 
[[Category:Core]]

Revision as of 21:38, 13 May 2017

This article is a work in progress.

IPython is so much more useful than the typical python shell, that the first thing to do as a programmer on a new computer is to install IPython before anything else! IPython gives you features like tab completion, running programs from the shell, editing files from inside python, saving history, and exploring source code among other things. Version 6 of IPython now also has static completion, so if you open square brackets you can tab-complete Python lists and dictionaries, write really long commands and use fewer temporary variables.

Don't worry if you have never programmed with Python before. The language is fun to learn. Learning python is beyond the scope of this guide, so feel free to consult the resources in the sidebar of the Python subreddit. If you need help and are new to programming in general, you can post to Learn Python or contact the HPC admins if your question is cluster related.

Requirements for this guide:

  • Some experience with the shell
  • Some experience with any programming language

Learning objectives:

  • Overview of IPython
  • Building up a script
  • Running python programs
  • Debugging errors

This introduction to IPython is part of the HPC intermediate workshop.

Install IPython

First check the version of Python you are currently using.

python --version
#> Python 2.6.6

The operating system version of Python 2.6.6 is ancient, so if that's the version you have by default, you should load a more recent version. We need Python 3.4 or later for IPython 6's static code completion, so let's use the latest python 3.6.1:

module -t avail python
module load python/3.6.1
python --version
#> Python 3.6.1

I promise that is the last time we will use python ^_~

Now we can install IPython:

pip3 install --user ipython
#> Successfully installed decorator-4.0.11 ipython-6.0.0 ipython-genutils-0.2.0 jedi-0.10.2 pexpect-4.2.1 pickleshare-0.7.4 prompt-toolkit-1.0.14 ptyprocess-0.5.1 pygments-2.2.0 simplegeneric-0.8.1 traitlets-4.3.2 wcwidth-0.1.7

The --user argument is to install the packages to our home directory under ~/.local/ because we don't have write permissions to the system directory /apps2/python/3.6.1/.

Now IPython is installed! But if we try to run ipython we get an error because it is not yet in the $PATH:

ipython
#> -bash: ipython: command not found

Let's edit our ~/.bashrc so that the shell knows where to search for the program:

echo 'export PATH=$HOME/.local/bin:$PATH' >> ~/.bashrc
source ~/.bashrc
which ipython
#> ~/.local/bin/ipython

Run ipython:

ipython
#> Python 3.6.1 (default, May 13 2017, 20:28:40) 
#> Type 'copyright', 'credits' or 'license' for more information
#> IPython 6.0.0 -- An enhanced Interactive Python. Type '?' for help.
#> 
#> In [1]:

Overview

Spent 2 minutes to read the builtin help as suggested by the startup message:

?

Like many IPython commands, the help opens up inside of the less pager. To navigate, use your the keyboard commands for less you know and love:

Keyboard Shortcut Action
arrow keys Move one line
space Next page
b Previous page
g Jump to top
G Jump to bottom
/word Search for "word"
n Next search result
N Previous search result
q Quit
h Help

As the help screen explains, IPython gives us all the power of the shell inside of python. We can run shell commands inside of Python, so that we never heave to leave python to run various housekeeping commands. This is possible using what it calls "magic". All magic commands begin with a % including the command to learn about "magic"!

Exit out of the help screen and let's read the short form of magic commands:

%quickref

One can see the equivalent full help with %magic

%magic

As you can see the help is vast. For now let's look at the few most important commands by creating our project directories.

# The [TAB] here means use the TAB key; don't actually type "[TAB]" ^_~

cd /scr[TAB]
#> cd /scratch/
cd /scratch/
pwd
#> '/gpfs/scratchfs1'
ls
#> (Lots of directories!)
mkdir -p abc12345/ipy # Replace with your NetID!
cd abc12345/ipy       # Replace with your NetID!
#> /gpfs/scratchfs1/abc12345/ipy
mkdir data src results
ls
#> data/  results/  src/
!tree $PWD
#> /gpfs/scratchfs1/abc12345/ipy
#> ├── data
#> ├── results
#> └── src
#> 
#> 3 directories, 0 files

To run any shell command, we put an exclamation mark ! before the command name. We used the !tree command to recursively print all out directores in a nice human readable format. Here $PWD is a shell variable which we are able to access inside of IPython when running shell commands with !. Strictly speaking, the $PWD was not necessary, but it shows the tree with the absolute path and also demonstrates that shell variables are usable with shell commands.

Now that we have our directory structure we can move on to the next section of creating our Python script.