cs501r_f2017:lab04
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
cs501r_f2017:lab04 [2017/09/16 17:13] humphrey [Objective] |
cs501r_f2017:lab04 [2021/06/30 23:42] (current) |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ====Objective==== | ====Objective==== | ||
| - | The purpose of this lab is to help students to understand the basic workflow for tensorflow. It’s composed of six parts: 1. Understand sessions, placeholders, and computation graphs; 2. Understand variables in tensorflow, and be able to train them with a simple delta rule; 3. Understand vectors, matrices and tensors, and be able to translate them into numpy arrays; 4. Understand data importation in python and tensorflow, be able to import a simple csv file and learn its hidden parameters; 5.(extra credit) Understand OOP in python, magic functions, decorator patent, and be able to express a machine learning model in terms of an object oriented language. | + | The purpose of this lab is to help students understand the basic workflow for tensorflow. It’s composed of five parts: 1. Understand sessions, placeholders, and computation graphs; 2. Understand variables in tensorflow, and be able to train them with a simple delta rule; 3. Understand vectors, matrices and tensors, and be able to translate them into numpy arrays; 4. Understand data importation in python and tensorflow, be able to import a simple csv file and learn its hidden parameters; 5.(extra credit) Understand OOP in python, magic functions, decorator patent, and be able to express a machine learning model in terms of an object oriented language. |
| After this lab, students should understand the basic workflow of building and testing a machine learning model in tensorflow, and be able to leverage on what them have learnt to build any machine learning model after they have sufficient understanding on its inner mechanics. | After this lab, students should understand the basic workflow of building and testing a machine learning model in tensorflow, and be able to leverage on what them have learnt to build any machine learning model after they have sufficient understanding on its inner mechanics. | ||
| In this lab, we are going to write a simple perceptron for linear regression. | In this lab, we are going to write a simple perceptron for linear regression. | ||
| Line 7: | Line 7: | ||
| ---- | ---- | ||
| ====Deliverable==== | ====Deliverable==== | ||
| - | Finish task 1 to 5, zip up all your code and ipython script together with the result of task 4, which is a tuple of 3 numbers (w1, w2, b), then submit that on learning suit. 😊 | + | Finish task 1 to 5, zip up all your code and ipython script together with the result of task 4, which is a tuple of 3 numbers (w1, w2, b), then submit that on learning suit. Task 5(extra credit) worth 5% of the total grade of this lab.😊 |
| + | |||
| + | ---- | ||
| + | |||
| + | ====Grading standards:==== | ||
| + | |||
| + | Your code will be graded on the following: | ||
| + | |||
| + | * 30% Correct implementation of data generator | ||
| + | * 30% Correct implementation of regression estimator | ||
| + | * 10% Correct implementation of multi values regression estimator | ||
| + | * 10% Fully vectorized code | ||
| + | * 20% Correct estimation of hidden parameters in foo.csv | ||
| + | * +5% Clean factorization of computation graphs into classes | ||
| ---- | ---- | ||
| Line 13: | Line 26: | ||
| ===Key concepts:=== | ===Key concepts:=== | ||
| - | Computation graphs: A computation graph is essentially an electric circuit, or you can think of it as a dynamical system if you are a math student. Given that, there are three things we want to do with it: first, feed it with some inputs; second, measure the readings of its output nodes; third, trigger some operations on occasions for more control on the graph/circulatory/system. | + | Computation graphs: A computation graph is essentially an electric circuit, or you can think of it as a dynamical system if you are a math student. Given that, there are three things we would like to do with it: first, feed it with some inputs; second, measure the readings of its output nodes; third, trigger some operations on occasions for more control on the system. |
| - | Sessions: It provide a framework to send and read signals to/from a graph, and it has very similarly syntax as a file stream. | + | Sessions: It provides a framework to send and read signals to or from a graph, and it has very similarly syntax as a file stream. |
| Placeholders: They are the input ports of a graph. Each time we run a computation graph with the goal of triggering an operation or measuring a set of nodes, it’s required to send in the request with an input dictionary, specifying what input values are used to generate the outputs. | Placeholders: They are the input ports of a graph. Each time we run a computation graph with the goal of triggering an operation or measuring a set of nodes, it’s required to send in the request with an input dictionary, specifying what input values are used to generate the outputs. | ||
| Line 65: | Line 78: | ||
| ====Understand Variables In Tensorflow==== | ====Understand Variables In Tensorflow==== | ||
| ===Key Concept:=== | ===Key Concept:=== | ||
| - | Variables: Variables are mutable state values in a computation graph, we typically want to setup a computation scheme to learn them with given data. | + | Variables: Variables are mutable state values in a computation graph, we typically want to setup a computation scheme to learn them with given data. In the language of perceptron, we also call them trainable weights. |
| Delta rule: Delta rule is one of the most rudimentary learning scheme we have in machine learning. It’s formulated as follow: | Delta rule: Delta rule is one of the most rudimentary learning scheme we have in machine learning. It’s formulated as follow: | ||
| δw_i=c(t - net) * x_i, | δw_i=c(t - net) * x_i, | ||
| - | where delta w_i stand for the change of weight i, c stand for the learning rate, t stand for the target value, net stand for the net sum of all weighted references and x_i is the value of the reference point x at position i. The intuition is that every time the guess of weights fail to predict the target value of a reference x, the algorithm will make small corrections towards the right direction. | + | where delta w_i stand for the change of weight i, c stand for the learning rate, t stand for the target value, net stand for the sum of all weighted features(in other words, dot products between weights and features), and x_i stands for input features. The intuition is that every time the guess of weights fail to predict the target value of a reference x, the algorithm will make small corrections towards the right direction. If you are familiar with perceptron rule in lab 02, then you may realize that we just change prediction to net in the equation. |
| ===Related readings:=== | ===Related readings:=== | ||
| Line 117: | Line 130: | ||
| ===Hints:=== | ===Hints:=== | ||
| - | 1. Use tf.assign to update variables on a computation graph | + | 1. Use tf.assign to update variables on a computation graph, we will use other optimization operators in the future, but they are just fancy version of tf.assign. |
| 2. Use tf.global_variables_initializer() to initialize variables | 2. Use tf.global_variables_initializer() to initialize variables | ||
| - | + | ||
| 3. You can run multiple ops in parallel by putting them into a list in sess.run like the following: | 3. You can run multiple ops in parallel by putting them into a list in sess.run like the following: | ||
| <code python> | <code python> | ||
| Line 126: | Line 139: | ||
| </code> | </code> | ||
| - | 4. My computation graph visualization looks like this | + | 4. My computation graph visualization looks like the following: |
| + | {{:cs501r_f2017:hint2.4.png?800|}} | ||
| ---- | ---- | ||
| ====Understand vectors, matrices and tensors==== | ====Understand vectors, matrices and tensors==== | ||
| Line 181: | Line 194: | ||
| ---- | ---- | ||
| ====Understand data importation in python and tensorflow==== | ====Understand data importation in python and tensorflow==== | ||
| - | There isn’t any secret about data importation after we understand what placeholders are in tensorflow. We just need to read data in efficiently enough that it’s not hindering the speed of our learning. We will work with images and natural languages later in the semester which will require more advance data pre-processing. However, let’s focus on the basics for now. We have included a link below to introduce saving and retrieving computation graph for those of you who are interested. | + | There isn’t any secret about data importation after we understand what placeholders are in tensorflow. We just need to read data in efficiently enough that it’s not hindering the speed of learning. We will work with images and natural languages later in this semester which will require more advance data pre-processing. However, let’s focus on the basics for now. We have included a link below to introduce saving and retrieving computation graph for those of you who are interested. |
| ===Related reading:=== | ===Related reading:=== | ||
| Line 189: | Line 202: | ||
| ===Task 4=== | ===Task 4=== | ||
| - | ==Read in the following csv file and guess the regression line behind the data.== | + | ==Read in the following .csv file and guess the regression line behind the data.== |
| + | [[cs501r_f2017:lab04:foo|foo.csv]] | ||
| ===Hints:=== | ===Hints:=== | ||
| Line 196: | Line 209: | ||
| ---- | ---- | ||
| - | ====Understand OOP in Tensorflow==== | + | ====Understand OOP in Tensorflow(Extra credit)==== |
| - | Object oriented programming(OOP) is a powerful tool to better understand the inherent structure of our code, and using it seamlessly with Tensorflow will give us more control on what do we want to accomplish. | + | Object oriented programming(OOP) is a powerful tool to better structure our code, and using it seamlessly with tensorflow will give us more control on designing new ML algorithms. Though this part of the lab is designed to be an extra credit exercise, it is highly recommended for all CS students to participate. It will only take you at most 30 minutes to finish if you are already familiar with OOP in python. |
| ===Key ideas:=== | ===Key ideas:=== | ||
| + | Running ops vs creating ops: Every line of code describing some operations in tensorflow is essentially creating nodes and linking them to the rest of a computation graph. Hence putting tensorflow code into functions and calling them for each run will result in duplicated definition of variables. Now, the question we want to answer in this session is that how can we organize our computations into logical block of codes, namely functions and classes in OOP, while avoiding the flaw of mixing up processes of creating and running ops. | ||
| + | |||
| + | magic methods: They are methods of a class with double underscore before and after the definition of its name. The magic behind them is that their invocations are implicitly defined for all objects. For example, if we want to write a constructor for your class, you can simply override the __init__ method. | ||
| + | |||
| + | decorator pattern: A decorator function takes a function and its arguments then extend its behavior without changing the function's implementation. The @annotation in python does exactly that. | ||
| + | |||
| + | @property annotation: While all variables in an object are visible to its users, we might still want to implement getters and setters with special behaviors, for example, bounds checking. The @property allows us to do exactly that. | ||
| ===Related reading:=== | ===Related reading:=== | ||
| https://danijar.com/structuring-your-tensorflow-models/ | https://danijar.com/structuring-your-tensorflow-models/ | ||
| + | https://www.learnpython.org/en/Classes_and_Objects | ||
| + | https://docs.python.org/3/reference/datamodel.html#special-method-names | ||
| + | https://www.thecodeship.com/patterns/guide-to-python-function-decorators/ | ||
| + | https://www.programiz.com/python-programming/property | ||
| Line 213: | Line 237: | ||
| 1. I got the following computation graph which are scoped by functions | 1. I got the following computation graph which are scoped by functions | ||
| + | {{:cs501r_f2017:hint5.1.png?800|}} | ||
| + | |||
| + | 2. The article by Danijar have shed great insight about this topic. Solutions to the problem should become trivial after reading his article. | ||
| + | |||
| + | 3. @functools.wraps(function) can be replaced by @six.wraps(function) for python 2 compatibility after installing and importing the python library "six" in your project environment. | ||
cs501r_f2017/lab04.1505581990.txt.gz · Last modified: 2021/06/30 23:40 (external edit)
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
