Introduction to Python Part 2 - PowerPoint Presentation

Slide1

Introduction to PythonPart 2 v0.6

Research Computing Services

Information Services & Technology

Slide2

Tutorial Outline – Part 2ListsTuples and dictionaries

Modules

numpy

and matplotlib modulesScript setup Development notes

Slide3

ListsA Python list is a general purpose 1-dimensional container for variables.

i.e. it is a row, column, or vector of things

Lots of things in Python act like lists or use list-style notation.

Variables in a list can be of any type at any location, including other lists.Lists can change in size: elements can be added or removed

Slide4

Making a list and checking it twice…Make a list with [ ] brackets.

Append with the

append()

functionCreate a list with some initial elementsCreate a list with N repeated elements

Try these out yourself!

Edit the file in

Spyder

and run it.

Add some print() calls to see the lists.

Slide5

List functionsTry dir

(list_1)

Like strings, lists have a number of built-in functions

Let’s try out a few…Also try the len() function to see how many things are in the list:

len

(list_1)

Slide6

List IndexingElements in a list are accessed by an index number.

Index #’s start at 0.

List

:

First element:

x[0]



'a'

Nth element:

x[2]



'c'

Last element:

x[-1

] 'e'Next-to-last: x[-2] 'd'

x

=[

'a'

,

'b'

,

'c'

,

'd'

,

'e'

]

Slide7

List SlicingSlice syntax:

x[

start:end:step

]

The start value is inclusive, the end value is exclusive.

Start is optional and defaults to 0.

Step is optional and defaults to 1.

Leaving out the end value means “go to the end”

Slicing always returns a

new list copied from the existing list

x

=[

'a'

,

'b', 'c', 'd'

,

'e

'

]

x

[

0

:

1

]



[

'a'

]

x

[

0

:

2

]



[

'

a'

,

'b

'

]

x

[-

3

:]



[

'c'

,

'd'

,

'e'

]

# Third from the end to the end

x

[

2

:

5

:

2

]



[

'c'

,

'e'

]

Slide8

List assignments and deletionsLists can have their elements overwritten or deleted (with the

del)

command

.

x

=[

'a'

,

'b'

,

'c'

,

'd' ,'e']x[0

]

=

-

3.14



x

is

now

[-

3.14

,

'b'

,

'c'

,

'd'

,

'e'

]

del

x

[-

1

]



x

is

now

[-

3.14

,

'b'

,

'c'

,

'd'

]

Slide9

DIY ListsIn the Spyder editor try the following things:

Assign some lists to some variables. a = [1,2,3] b = 3*[‘xyz’]

Try an empty list, repeated elements, initial set of elements

Add two lists: a + b What happens?Try list indexing, deletion, functions from

dir

(

my_list

)

Try assigning the result of a list slice to a new variable

Go to the menu

File

New

File

Enter your list commands there

Give the file a name when you save it

Use print() to print out results

Slide10

More on Lists and VariablesOpen the sample file list_variables.py

but don’t run it yet!

What do you think will be printed?

Slide11

Variables and Memory LocationsVariables refer to a value stored in memory.

y = x

does

not mean “make a copy of the list x and assign it to y” it means “make a copy of the memory location in x and assign it to y”

x is

not the list

it’s just a reference to it.

This is how all objects in Python are handled.

x

y

Slide12

Copying ListsHow to copy (2 ways…there are more!):

y

= x

[:]

or

y=list(x

)

In

list_variables.py

uncomment the code at the bottom and run it.

Slide13

While LoopsWhile loops have a condition and a code block.

the indentation indicates what’s in the while loop.

The loop runs until the condition is false.

The break keyword will stop a while loop running.

In the

Spyder

edit enter in some loops like these. Save and run them one at a time. What happens with the 1

st

loop?

Slide14

For loopsfor loops are a little different. They loop through a collection of things.

The for loop syntax has a collection and a code block.

Each element in the collection is accessed in order by a reference variable

Each element can be used in the code block.

The

break

keyword can be used in for loops too.

collection

In-loop reference variable for each collection element

The code block

Slide15

Processing lists element-by-elementA for loop is a convenient way to process every element in a list.

There are several ways:

Loop over the list elements

Loop over a list of index values and access the list by indexDo both at the same timeUse a shorthand syntax called a

list comprehension

Open the file

looping_lists.py

Let’s look at code samples for each of these.

Slide16

The range() functionThe range() function auto-generates sequences of numbers that can be used for indexing into lists.Syntax: range(

start

,

exclusive end, increment)range(0,4)



produces the sequence of numbers 0,1,2,3

range(-3,15,3)

 -3,0,3,6,9,12

range(4,-3,2)

 4,2,0,-2Try this: print(range(4))

Slide17

Lists With LoopsOpen the file read_a_file.py

This is an example of reading a file into a list. The file is shown to the right,

numbers.txt

We want to read the lines in the file into a list of strings (1 string for each line), then extract separate lists of the odd and even numbers.

Let’s walk through this line-by-line using

Spyder

read_a_file_low_mem.py

is a modification that uses less memory.

38,83,37,21,98

50,53,55,37,97

39,7,81,87,82

18,83,66,82,47

56,64,9,39,83

…etc…

numbers.txt

Slide18

Tutorial Outline – Part 2ListsTuples and dictionaries Modules

numpy

and

matplotlib modulesScript setup Development notes

Slide19

TuplesTuples are lists whose elements can’t be changed.Like strings they are immutable

Indexing (including slice notation) is the same as with lists.

Slide20

Return multiple values from a functionTuples are more useful than they might seem at first glance.

They can be easily used to return multiple values from a function.

Python syntax can automatically unpack a tuple return value.

Slide21

DictionariesDictionaries are another basic Python data type that are tremendously useful.Create a dictionary with a pair of curly braces:

x

=

{}

Dictionaries store

values

and are indexed with

keys

Create a dictionary with some initial values:

x

=

{

'a_key':55, 100:'a_value',

4.1

:[

5

,

6

,

7

]}

Slide22

DictionariesValues can be any Python thing

Keys can be primitive types (numbers), strings, tuples, and some custom data types

Basically, any data type that is

immutable

Lists and dictionaries cannot be keys but they can stored as values.

Index dictionaries via keys:

x

[

'

a_key

'

]



55x[100]



'

a_value

'

Slide23

Try Out DictionariesCreate a dictionary in the Python console or

Spyder

editor.

Add some values to it just by using a new key as an index. Can you overwrite a value?

Try

x.keys

()

and

x.values

()

Try:

del x[

valid_key

]

 deletes a key/value pair from the dictionary.

Slide24

Tutorial Outline – Part 2ListsTuples and dictionaries Modules

numpy

and

matplotlib modulesScript setup Development notes

Slide25

ModulesPython modules, aka libraries or packages, add functionality to the core Python language.The

Python Standard Library

provides a very wide assortment of functions and data structures.

Check out their Brief Tour for a quick intro.

Distributions like Anaconda provides dozens or hundreds more

You can write your own libraries or install your own.

Slide26

PyPIThe Python Package Index is a central repository for Python software.

Mostly but not always written in Python.

A tool,

pip, can be used to install packages from it into your Python setup.Anaconda provides a similar tool called

conda

Number of projects (as of January 2019):

164,947

You should always do your due diligence when using software from a place like

PyPI

. Make sure it does what you think it’s doing!

Slide27

Python Modules on the SCCPython modules should not be confused with the SCC module command.

For the SCC there are

instructions

on how to install Python software for your account or project.Many SCC modules provide Python packages as well.Example:

tensorflow

,

pycuda

, others.

Need help on the SCC? Send us an email:

help@scv.bu.edu

Slide28

Importing modulesThe import

command is used to load a module.

The name of the module is prepended to function names and data structures in the module.

The preserves the module namespaceThis allows different modules to have the same function names – when loaded the module name keeps them separate.

Try these out!

Slide29

Fun with importThe

import

command can strip away the module name:

Or it can import select functions:

Or rename on the import:

from

math

import

*

from

math

import

cosfrom math import cos,sqrt

from

math

import

sin

as

pySin

Slide30

Easter Eggs

from

__future__

import

braces

i

mport

antigravity

Slide31

Fun with importThe

import

command can also load your own Python files.

The Python file to the right can be used in another Python script:

def

get_odds

(

lst

):

''' Gets the odd numbers in a list.

lst

: incoming list of integers return: list of odd integers ''' odds =

[]

for

elem

in

lst

:

# Odd if there's a remainder when

# dividing by 2.

if

elem

%

2

!=

0

:

odds

.

append

(

elem

)

return

odds

myfuncs.py

# Don't use the .

py

ending

import

myfuncs

x

=

[

1

,

2

,

3

,

4

]

y

=

myfuncs

.

get_odds

(

x

)

Slide32

Import detailsPython reads and executes a file when the fileis opened directly:

python somefile.py

is

imported:

import

somefile

Lines that create variables, call functions, etc. are all executed.

Here these lines will run when it’s imported into another script!

def

get_odds

(

lst

):

''' Gets the odd numbers in a list. lst: incoming list of integers

return: list of odd integers '''

odds

=

[]

for

elem

in

lst

:

# Odd if there's a remainder when

# dividing by 2.

if

elem

%

2

!=

0

:

odds

.

append

(

elem

)

return

odds

x

=

[

1

,

2

,

3

,

4

]

y

=

get_odds

(

x

)

print

(

y

)

myfuncs.py

Slide33

The __name__ attributePython stores object information in hidden fields called attributes

m

yfuncs.py

# in another Python

# script

import

myfuncs

Every file has one called __name__ whose value depends on how the file is used.

__name__



myfuncs

(i.e. the file name)

# called directly

python myfuncs.py

__name__



__main__

Slide34

The __name__ attribute__name__ can be used to make a Python scripts usable as a standalone program

and

as imported code.

Now: python myfuncs.py

 __name__ has the value of ‘__main__’ and the code in the

if

statement is executed.

import

myfuncs

 __name__ is ‘

myfuncs

’ and the

if

statement does not run.

def get_odds(lst): ''' Gets the odd numbers in a list.

lst

: incoming list of integers

return: list of odd integers '''

odds

=

[]

for

elem

in

lst

:

# Odd if there's a remainder when

# dividing by 2.

if

elem

%

2

!=

0

:

odds

.

append

(

elem

)

return

odds

if

__name__

==

'__main__'

:

x

=

[

1

,

2

,

3

,

4

]

y

=

get_odds

(

x

)

print

(

y

)

myfuncs.py

Slide35

Tutorial Outline – Part 2ListsTuples and dictionaries Modules

numpy

and

matplotlib modulesScript setup Development notes

Slide36

A brief into to numpy and matplotlib

numpy

is a Python library that provides efficient multidimensional matrix and basic linear

algrebraThe syntax is very similar to Matlab or Fortran

matplotlib

is a popular plotting library

Remarkably similar to

Matlab

plotting commands!

A third library, scipy, provides a wide variety of numerical algorithms:Integrations, curve fitting, machine learning, optimization, root finding, etc.

Built on top of

numpy

Investing the time in learning these three libraries is worth the effort!!

Slide37

numpynumpy provides data structures written in compiled C code

Many of its operations are executed in compiled C or Fortran code, not Python.

Check out

numpy_basics.py

Slide38

numpy datatypesUnlike Python lists, which are generic containers,

numpy

arrays are typed.

If you don’t specify a type, numpy will assign one automatically.A

wide variety of numerical types

are available.

Proper assignment of data types can sometimes have a significant effect on memory usage and performance.

Slide39

Numpy operatorsNumpy

arrays will do element-wise arithmetic: + / - * **

Matrix (or vector/matrix, etc.) multiplication needs the .dot() function.

Numpy has its own sin(), cos(), log(), etc. functions that will operate element-by-element on its arrays.

Try these out!

Slide40

Plotting with matplotlibMatplotlib

is probably the most popular Python plotting library

Plotly

is another good oneIf you are familiar with Matlab plotting then

matplotlib

is very easy to learn!

Plots can be made from lists, tuples,

numpy

arrays, etc.

Try these out!

Slide41

Some sample images from matplotlib.orgA vast array of plot types in 2D and 3D are available in this library.

Slide42

A numpy and matplotlib example

numpy_matplotlib_fft.py

is a short example on using

numpy and matplotlib together.Open

numpy_matplotlib_fft.py

Let’s walk through this…

Slide43

Tutorial Outline – Part 2ListsTuples and dictionaries Modules

numpy

and

matplotlib modulesScript setup Development notes

Slide44

Writing Quality Pythonic Code

Cultivating good coding habits pays off in many ways:

Easier and faster to write

Easier and faster to edit, change, and update your codeOther people can understand your workPython lends itself to readable code

It’s quite hard to write

completely

obfuscated code in Python.

Exploit language features where it makes sense

Contrast that with

this sample of obfuscated C code.Here we’ll go over some suggestions on how to setup a Python script, make it readable, reusable, and testable.

Slide45

Compare some Python scriptsOpen up three files and let’s look at them.A file that does…something…

bad_code.py

Same code, re-organized:

good_code.pySame code, debugged, with testing code:good_code_testing.py

Slide46

Command line argumentsTry to avoid hard-coding file paths, problem size ranges, etc. into your program.

They can be specified at the command line.

Look at the

argparse module, part of the Python Standard Library.

Slide47

Tutorial Outline – Part 2ListsTuples and dictionaries

Modules

numpy

and matplotlib modulesScript setup Development notes

Slide48

Function, class, and variable namingThere’s no word or character limit for names.It’s ok to use descriptive names for things.

An

IDE (like

Spyder) will help you fill in longer names so there’s no extra typing anyway.Give your functions and variables names that reflect their meaning.

Once a program is finished it’s easy to forget what does what where

Slide49

An example development processWork to develop your program.

Do some flowcharts, work out algorithms, and so on.

Write some Python to try out a few ideas.

Get organized.Write a “1st draft” version that gets most of what’s needed done.

Move

hard-coded values into the

if __name__==‘__main__’

section of your code

.

Once the code is testing well add command line arguments and remove hard-coded values

Finally (e.g. to run as an SCC batch job) test run from the command line.

Slide50

Spyder command line argumentsClick on the Run menu and choose Configuration per file

Enter command line arguments

Slide51

Python from the command lineTo run Python from the command line:

Just type

python

followed by the script name followed by script arguments.

Slide52

Where to get help…The official Python Tutorial

Automate the Boring Stuff with Python

Focuses more on doing useful things with Python, not focused on scientific computing

Full Speed Python tutorial

Contact Research Computing:

help@scv.bu.edu