View source for Python/Miscellaneous Snippets and Tips

Here are miscellaneous tips and tricks when working with Python files.

== Migrating from MATLAB ==
Numpy (the defacto numerical array library in Python) has a handy [https://numpy.org/doc/stable/user/numpy-for-matlab-users.html guide for migrating from MATLAB to Numpy].

Here are some tips for general Python:

=== Indexing ===
==== Python indices start with 0 instead of 1 ====
This has several cascading effects in the language, such as:
 >>> test = list(range(3))
 >>> print(test) # Doesn't include 3, but creates 3 integers
 [0, 1, 2]
 >>> test[0:2] # Excludes the last index, so only 2 integers are returned
 [0, 1]

==== <code>[...]</code> vs <code>(...)</code> after an object ====
Say you have <code>object[0]</code> or <code>object['foo']</code> or <code>object('foo')</code>. What can you tell about <code>object</code> without looking at it's definition? Quite a lot actually!

In general: 
* <code>[...]</code> gets a value from a Python object
** Either by [https://docs.python.org/3/reference/expressions.html#subscriptions subscription] (<code>object[0]</code> or <code>object['asdf']</code>) or [https://docs.python.org/3/reference/expressions.html#slicings slicing] (<code>object[1:3]</code>)
* <code>(...)</code> declares arguments to a function (like <code>range(3)</code>).

'''Examples of getting a value from a Python object:'''

 >>> lista = ['a', 'b', 3]
 >>> lista[0]    # Accessing item 0
 'a'
 
 >>> lista[1:]   # Accessing all items of index 1 or greater
 ['b', 3]
 
 >>> lista[0] = (1, 3, 'asdf')   # Assigning a tuple to the 0th element
 >>> lista
 [(1, 3, 'asdf'), 'b', 3]
 
 >>> lista[0]    # Accessing the 0th item (the tuple)
 (1, 3, 'asdf')
 
 >>> lista[0][2]    # Accessing nested lists
 'asdf'

 >>> dicta = {'a': 2, 'b':'foo', 'bar':{'sub':'phasta'}}
 >>> dicta['a']   # Accessing the value of the key 'a'
 2
 >>> dicta['bar']  # Accessing the value of 'bar', a nested dictionary
 {'sub': 'phasta'}
 >>> dicta['bar']['sub'] # Accessing nested dictionary items
 'phasta'

== Write Data to Text File (ie. CSV) ==
Given data in some Python object (most likely a numpy-derived array, but possibly just a normal Python list), how do you write it out to a file? [https://numpy.org/doc/stable/reference/generated/numpy.savetxt.html '''Use <code>numpy.savetxt</code>'''] (or more likely <code>np.savetxt</code>). 

'''Example:''' Given a array, <code>A</code>, of shape <code>[n, m]</code>, simply use 
 np.savetxt('path/file.dat', A)

which creates a file with <code>n</code> rows and <code>m</code> columns.

[https://numpy.org/doc/stable/reference/generated/numpy.savetxt.html Numpy's documentation] has information on other useful arguments to change numerical formats, separators, and adding headers to the file.

=== Write multiple 1D arrays as columns ===
To do this, [https://numpy.org/doc/stable/reference/generated/numpy.column_stack.html use <code>numpy.column_stack</code>] to create an array with the columns "stacked" together. 

'''Example:''' Given two 1D arrays, <code>a</code> and <code>b</code>, of the same size, use:

 np.savetxt('path/file.dat', np.column_stack((a,b)) )

Two things to note here:
# <code>np.column_stack</code> takes a ''list'' or ''tuple'' as an argument, hence the two sets of <code>((...))</code>.
# <code>np.column_stack</code> creates an entirely new array and copies the given data into it. As such, it will double the total amount of memory used; once for the original 1D arrays, and again for the brand new array storing a copy of the original data.
#* If data format is flexible, consider [[Python/Miscellaneous_Snippets_and_Tips#Write_multiple_1D_arrays_as_rows|writing in rows]] instead of columns as it is much faster (~20%, no time spent copying data) and uses less memory

=== Write multiple 1D arrays as rows ===
<code>np.savetxt</code> will also take 2D-like array input. This means you can pass a list/tuple of arrays and it will process each array as a row.

'''Example:''' Given two 1D arrays, <code>a</code> and <code>b</code>, of the same size, use:

 np.savetxt('path/file.dat', (a,b) )

Note we do not need to invoke <code>np.column_stack</code>, and thus we don't spend time copying data or take up memory with redundant data.

== Running Python Files in Terminal ==

There are a few ways to run Python files (and Python code more generally): via a Unix shebang, <code>python</code>, inside a <code>ipython</code> instance, or through an IDE like <code>spyder</code>. We'll go over how to run a script, <code>script.py</code>, in the terminal:

 ~$ cat script.py
 print('Hello World')

''Note: I'll be assuming usage of Python 3.X. If using Python 2.7, use <code>python</code> instead of <code>python3</code>''
=== Unix Shebang ===
Using a [https://en.wikipedia.org/wiki/Shebang_(Unix) Unix shebang], you can execute the script directly in a Linux terminal (if it has the appropriate execute permissions). 
This is done by adding a <code>#!/usr/bin/env python3</code> to the top of the file.
This line is not read by Python (<code>#</code> is the comment character), but instead by the (Linux) kernel which uses the executable specified in the line to execute the file.
Then you can execute the file directly.

 ~$ cat script.py
 #!/usr/bin/env python3
 print('Hello World')
 
 ~$ ./script.py
 Hello World

Adding a shebang does not exclude you from running the script in any of the other methods listed below; Python will simply ignore the line since it is a commented line.

=== Default Python Interpreter ===
You can execute <code>script.py</code> by calling the script as an argument to the python interpreter executable, much like you can do with bash, zsh, or perl:

 ~$ python3 script.py
 Hello World

=== IPython Console ===
The [https://ipython.readthedocs.io/en/stable/ IPython Console] is a very powerful, interactive python shell (ie. console or terminal) that is built into many other applications, such as Spyder and JupyterLab. It offers a host of useful features, tab completion, like system shell commands (<code>cd</code>, <code>ls</code>, etc), debugging shell, and also has special [https://ipython.readthedocs.io/en/stable/interactive/magics.html "magic" commands]. One of those magic commands is <code>%run</code>.

<code>%run</code> allows you to run a script and then interact with the variables that are created in the script, similarly to Matlab's console. This is especially useful when debugging a script; if your script outputs an error and stop, you can inspect the variable states right when the error occurred. 

'''''Note:''' IPython is not installed in default python installations. It is included in [[Anaconda]] installations and can also be installed quite easily via <code>pip</code> (<code>pip install ipython</code>) or <code>conda</code> (<code>conda install ipython</code>).''

 ~$ cat script.py
 #!/usr/bin/env python3
 print('Hello World')
 test = 1
 
 ~$ ipython
 Python 3.9.6 (default, Jun 30 2021, 10:22:16) 
 Type 'copyright', 'credits' or 'license' for more information
 IPython 7.23.1 -- An enhanced Interactive Python. Type '?' for help.
 
 In [1]: %run script.py
 Hello World
 
 In [2]: test
 Out[2]: 1

Note this can also be done with Python's default console, but it's a bit more clunky:

 ~$ python 
 Python 3.9.6 (default, Jun 30 2021, 10:22:16) 
 [GCC 11.1.0] on linux
 Type "help", "copyright", "credits" or "license" for more information.
 >>> exec(open('script.py').read())
 Hello World
 >>> test
 1

However Python's built-in console does not have the nice features of IPython, so it is generally preferred.