cs501r_f2017:lab04
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
cs501r_f2017:lab04 [2017/09/16 17:51] humphrey [Understand Variables In Tensorflow] |
cs501r_f2017:lab04 [2021/06/30 23:42] (current) |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ====Objective==== | ====Objective==== | ||
| - | The purpose of this lab is to help students to understand the basic workflow for tensorflow. It’s composed of six parts: 1. Understand sessions, placeholders, and computation graphs; 2. Understand variables in tensorflow, and be able to train them with a simple delta rule; 3. Understand vectors, matrices and tensors, and be able to translate them into numpy arrays; 4. Understand data importation in python and tensorflow, be able to import a simple csv file and learn its hidden parameters; 5.(extra credit) Understand OOP in python, magic functions, decorator patent, and be able to express a machine learning model in terms of an object oriented language. | + | The purpose of this lab is to help students understand the basic workflow for tensorflow. It’s composed of five parts: 1. Understand sessions, placeholders, and computation graphs; 2. Understand variables in tensorflow, and be able to train them with a simple delta rule; 3. Understand vectors, matrices and tensors, and be able to translate them into numpy arrays; 4. Understand data importation in python and tensorflow, be able to import a simple csv file and learn its hidden parameters; 5.(extra credit) Understand OOP in python, magic functions, decorator patent, and be able to express a machine learning model in terms of an object oriented language. |
| After this lab, students should understand the basic workflow of building and testing a machine learning model in tensorflow, and be able to leverage on what them have learnt to build any machine learning model after they have sufficient understanding on its inner mechanics. | After this lab, students should understand the basic workflow of building and testing a machine learning model in tensorflow, and be able to leverage on what them have learnt to build any machine learning model after they have sufficient understanding on its inner mechanics. | ||
| In this lab, we are going to write a simple perceptron for linear regression. | In this lab, we are going to write a simple perceptron for linear regression. | ||
| Line 7: | Line 7: | ||
| ---- | ---- | ||
| ====Deliverable==== | ====Deliverable==== | ||
| - | Finish task 1 to 5, zip up all your code and ipython script together with the result of task 4, which is a tuple of 3 numbers (w1, w2, b), then submit that on learning suit. 😊 | + | Finish task 1 to 5, zip up all your code and ipython script together with the result of task 4, which is a tuple of 3 numbers (w1, w2, b), then submit that on learning suit. Task 5(extra credit) worth 5% of the total grade of this lab.😊 |
| + | |||
| + | ---- | ||
| + | |||
| + | ====Grading standards:==== | ||
| + | |||
| + | Your code will be graded on the following: | ||
| + | |||
| + | * 30% Correct implementation of data generator | ||
| + | * 30% Correct implementation of regression estimator | ||
| + | * 10% Correct implementation of multi values regression estimator | ||
| + | * 10% Fully vectorized code | ||
| + | * 20% Correct estimation of hidden parameters in foo.csv | ||
| + | * +5% Clean factorization of computation graphs into classes | ||
| ---- | ---- | ||
| Line 13: | Line 26: | ||
| ===Key concepts:=== | ===Key concepts:=== | ||
| - | Computation graphs: A computation graph is essentially an electric circuit, or you can think of it as a dynamical system if you are a math student. Given that, there are three things we want to do with it: first, feed it with some inputs; second, measure the readings of its output nodes; third, trigger some operations on occasions for more control on the graph/circulatory/system. | + | Computation graphs: A computation graph is essentially an electric circuit, or you can think of it as a dynamical system if you are a math student. Given that, there are three things we would like to do with it: first, feed it with some inputs; second, measure the readings of its output nodes; third, trigger some operations on occasions for more control on the system. |
| - | Sessions: It provide a framework to send and read signals to/from a graph, and it has very similarly syntax as a file stream. | + | Sessions: It provides a framework to send and read signals to or from a graph, and it has very similarly syntax as a file stream. |
| Placeholders: They are the input ports of a graph. Each time we run a computation graph with the goal of triggering an operation or measuring a set of nodes, it’s required to send in the request with an input dictionary, specifying what input values are used to generate the outputs. | Placeholders: They are the input ports of a graph. Each time we run a computation graph with the goal of triggering an operation or measuring a set of nodes, it’s required to send in the request with an input dictionary, specifying what input values are used to generate the outputs. | ||
| Line 117: | Line 130: | ||
| ===Hints:=== | ===Hints:=== | ||
| - | 1. Use tf.assign to update variables on a computation graph | + | 1. Use tf.assign to update variables on a computation graph, we will use other optimization operators in the future, but they are just fancy version of tf.assign. |
| 2. Use tf.global_variables_initializer() to initialize variables | 2. Use tf.global_variables_initializer() to initialize variables | ||
| - | + | ||
| 3. You can run multiple ops in parallel by putting them into a list in sess.run like the following: | 3. You can run multiple ops in parallel by putting them into a list in sess.run like the following: | ||
| <code python> | <code python> | ||
| Line 126: | Line 139: | ||
| </code> | </code> | ||
| - | 4. My computation graph visualization looks like this | + | 4. My computation graph visualization looks like the following: |
| + | {{:cs501r_f2017:hint2.4.png?800|}} | ||
| ---- | ---- | ||
| ====Understand vectors, matrices and tensors==== | ====Understand vectors, matrices and tensors==== | ||
| Line 181: | Line 194: | ||
| ---- | ---- | ||
| ====Understand data importation in python and tensorflow==== | ====Understand data importation in python and tensorflow==== | ||
| - | There isn’t any secret about data importation after we understand what placeholders are in tensorflow. We just need to read data in efficiently enough that it’s not hindering the speed of our learning. We will work with images and natural languages later in the semester which will require more advance data pre-processing. However, let’s focus on the basics for now. We have included a link below to introduce saving and retrieving computation graph for those of you who are interested. | + | There isn’t any secret about data importation after we understand what placeholders are in tensorflow. We just need to read data in efficiently enough that it’s not hindering the speed of learning. We will work with images and natural languages later in this semester which will require more advance data pre-processing. However, let’s focus on the basics for now. We have included a link below to introduce saving and retrieving computation graph for those of you who are interested. |
| ===Related reading:=== | ===Related reading:=== | ||
| Line 189: | Line 202: | ||
| ===Task 4=== | ===Task 4=== | ||
| - | ==Read in the following csv file and guess the regression line behind the data.== | + | ==Read in the following .csv file and guess the regression line behind the data.== |
| + | [[cs501r_f2017:lab04:foo|foo.csv]] | ||
| ===Hints:=== | ===Hints:=== | ||
| Line 197: | Line 210: | ||
| ---- | ---- | ||
| ====Understand OOP in Tensorflow(Extra credit)==== | ====Understand OOP in Tensorflow(Extra credit)==== | ||
| - | Object oriented programming(OOP) is a powerful tool to better understand the inherent structure of our code, and using it seamlessly with tensorflow will give us more control on designing new ML algorithm. Though this part of the lab is designed to be an extra credit exercise, it is highly recommended for all CS students to participate. It will only take you at most 30 minutes to finish if you are already familiar with OOP in python. | + | Object oriented programming(OOP) is a powerful tool to better structure our code, and using it seamlessly with tensorflow will give us more control on designing new ML algorithms. Though this part of the lab is designed to be an extra credit exercise, it is highly recommended for all CS students to participate. It will only take you at most 30 minutes to finish if you are already familiar with OOP in python. |
| ===Key ideas:=== | ===Key ideas:=== | ||
| - | Running opts vs creating ops: | + | Running ops vs creating ops: Every line of code describing some operations in tensorflow is essentially creating nodes and linking them to the rest of a computation graph. Hence putting tensorflow code into functions and calling them for each run will result in duplicated definition of variables. Now, the question we want to answer in this session is that how can we organize our computations into logical block of codes, namely functions and classes in OOP, while avoiding the flaw of mixing up processes of creating and running ops. |
| - | decorator pattern: | + | magic methods: They are methods of a class with double underscore before and after the definition of its name. The magic behind them is that their invocations are implicitly defined for all objects. For example, if we want to write a constructor for your class, you can simply override the __init__ method. |
| - | @property annotation: | + | decorator pattern: A decorator function takes a function and its arguments then extend its behavior without changing the function's implementation. The @annotation in python does exactly that. |
| + | |||
| + | @property annotation: While all variables in an object are visible to its users, we might still want to implement getters and setters with special behaviors, for example, bounds checking. The @property allows us to do exactly that. | ||
| ===Related reading:=== | ===Related reading:=== | ||
| https://danijar.com/structuring-your-tensorflow-models/ | https://danijar.com/structuring-your-tensorflow-models/ | ||
| + | https://www.learnpython.org/en/Classes_and_Objects | ||
| + | https://docs.python.org/3/reference/datamodel.html#special-method-names | ||
| + | https://www.thecodeship.com/patterns/guide-to-python-function-decorators/ | ||
| + | https://www.programiz.com/python-programming/property | ||
| Line 218: | Line 237: | ||
| 1. I got the following computation graph which are scoped by functions | 1. I got the following computation graph which are scoped by functions | ||
| - | 2. The article by Danijar have shed great insight about the topic. Solution to the problem should become trivial after reading his article. | + | {{:cs501r_f2017:hint5.1.png?800|}} |
| + | |||
| + | 2. The article by Danijar have shed great insight about this topic. Solutions to the problem should become trivial after reading his article. | ||
| 3. @functools.wraps(function) can be replaced by @six.wraps(function) for python 2 compatibility after installing and importing the python library "six" in your project environment. | 3. @functools.wraps(function) can be replaced by @six.wraps(function) for python 2 compatibility after installing and importing the python library "six" in your project environment. | ||
cs501r_f2017/lab04.1505584284.txt.gz · Last modified: 2021/06/30 23:40 (external edit)
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
