TensorFlow Cheatsheet

General cheatsheet

Initialize a Variable using constants or random values (zeros, zeros_like, linspace, etc.).

Can save and restore models using tf.train.Saver.

Read on from here for more cool stuff: https://www.tensorflow.org/programmers_guide/threading_and_queues

Graph vs. Session

(following the great explanation by Danijar Hafner)
graph = tf.Graph()
with graph.as_default():
    v = tf.Variable(tf.random_normal(shape=[1]), name='foo')
    print v.shape, v.shape.ndims, v.shape.num_elements()  # (1,) 1 1
    if 1: # Don't do this! tf.global_variables_initializer() defines an op that initializes all variables in the graph (so far); so you should call this AFTER they were all defined, otherwise you'll get something like "FailedPreconditionError: Attempting to use uninitialized value FailedPreconditionError: Attempting to use uninitialized value foo_scalar"
        init_op_notgood = tf.global_variables_initializer()
    v_scalar = tf.Variable(tf.random_normal(shape=[]), name='foo_scalar'))  # shape=[] or shape=() defines a 0-dimensional tensor, i.e. scalar
    print v_scalar.shape, v_scalar.shape.ndims, v_scalar.shape.num_elements()  # () 0 1
    init_op = tf.global_variables_initializer()  # Add an op to initialize all variables in the graph (actually probably best to define this even further down after the entire graph was constructed, but defining it here is already okay for our example)
    
    assign_v = v.assign([101])
    assign_v_scalar = v_scalar.assign(102)

c = tf.constant(4.0)  # Will be defined as attached to the default graph (!) tf.get_default_graph()

# Sanity check
print c.graph == tf.get_default_graph(), variable.graph == graph, graph == tf.get_default_graph()  # True True False

Then instantiate a Session to run our graph:

with tf.Session(graph=graph) as sess:
    sess.run(init_op)
    print sess.run(v)  # e.g. [-0.407900009]
    print sess.run(v_scalar)  # e.g. 1.30248
    if 1:  # Don't do this part
        sess.run(init_op_notgood)
        print sess.run(v)  # e.g. [0.33414543], a different value than above
        print sess.run(v_scalar)  # e.g. 1.30248, same as above
    print sess.run([assign_v, assign_v_scalar])  # [array([ 101.], dtype=float32), 102.0] -- return values probably not really interesting here
    print sess.run(v)  # [ 101.]
    print sess.run(v_scalar)  # 102.0

    if 0: # Error, as 'c' is not an element of the current graph...
        print sess.run(c)  # ValueError: Fetch argument <tf.Tensor 'Const_50:0' shape=() dtype=float32> cannot be interpreted as a Tensor. (Tensor Tensor("Const_50:0", shape=(), dtype=float32) is not an element of this graph.)

Docker in Windows, allocate more memory

Some installation / cheatsheet

Using Docker for Windows, Windows 10. 17.03.0-ce-win1 (10296). Channel: stable. 94675c5.

(Tip: Also install Kitematic, then you can right-click the running Docker icon and from there launch Kitematic for a nice GUI to see your containers.)

Basic commands:

# Create a tensorflow container:
docker run -it -p 8888:8888 -p 6006:6006 --name my_tensor_flow -v C:/Data/Docker:/data -v C:/Dev/python:/devpython tensorflow/tensorflow
# Run it in the future:
docker start -ai my_tensor_flow
# To connect to it with bash: (note: /notebooks is where the notebooks are kept, and we can already access the host's C:/Data/Docker in /data)
docker exec -it my_tensor_flow bash
# Then from the bash can run: tensorboard --logdir=/data/tensorboard/5/

Apart from the nice GUI, there are some useful commands from the command line:

docker stats [--all]
docker ps [--all]
docker container list [--all]

Allocating more memory

For my running container, docker stats showed (in “MEM USAGE / LIMIT”) that it was bounded by 1.934 GiB. Indeed, “Hyper-V Manager” showed “MobyLinuxVM” machine has only 2GB.

To allocate more memory: Docker’s settings, Advanced, set Docker’s memory e.g. to 8448MB. Docker will restart. Verify that “Hyper-V Manager” now shows “MobyLinuxVM” has assigned memory of 8448 MB. Run your previous container docker start -ai container-name, and now docker stats will show a memory bound of 8.003 GiB. Success.

Note: Running some

docker run -it -p ... --name ... --memory-swap -1 --memory 8g -v ...
will not work if your “MobyLinuxVM” doesn’t have enough assigned memory.

Some notes about Entropy

KullbackÔÇôLeibler divergence (Wikipedia): A non-symmetric difference between two distributions:

    \[D_{KL}(P||Q) = \sum_i{P(i)\log\frac{P(i)}{Q(i)}}\]

Conditional Entropy:

    \[H(Y|X) = -\sum_{x}{p(x) \sum_{y}{p(y|x) \log p(y|x)} } = -\sum_{x,y}{p(x,y) \log \frac{p(x)}{p(x,y)}}\]

Joint Entropy:

    \[H(X,Y) = -\sum_{x,y}{p(x,y) \log p(x,y) }\]

    \[H(X,Y) = H(X) + H(Y|X) = H(Y) + H(X|Y)\]

A non-symmetric measure of association (“uncertainty coefficient”?) between X and Y measures the percentage of entropy reduced from Y if X is given:

    \[U(Y|X) = \frac{H(Y) - H(Y|X)}{H(Y)} = \frac{I(X;Y)}{H(Y)}\]

Or the percentage of entropy reduced from X if Y is given:

    \[U(X|Y) = \frac{H(X) - H(X|Y)}{H(X)} = \frac{I(X;Y)}{H(X)}\]

Where I(X;Y) = H(X) + H(Y) - H(X,Y) = H(X,Y) - H(X|Y) - H(Y|X), the mutual information of X and Y, is non-negative and symmetric.

Anyway, U(Y|X) equals 0 if no association, 1 if knowing X fully predicts Y (i.e. Y is a function of X).
A symmetric measure can be made of a weighted average of U(Y|X) and U(X|Y):

    \[U(X,Y) &= \frac{H(X)U(X|Y) + H(Y)U(Y|X)}{H(X)+H(Y)}\]

    \[= 2 \Big[ \frac{H(X)+H(Y)-H(X,Y)}{H(X)+H(Y)} \Big]\]

Relation to \chi^2 or measures like Cramer’s V etc.:

    \[\chi^2 = N \cdot \sum_{x,y}{\frac{(p(x,y)-p(x)p(y))^2}{p(x)p(y)}}\]

No obvious relation. Generated some 2×2 contigency tables and plotted their U(X,Y) vs Cramer’s V:

Running Apache and Node on the same server

Option 1. Apache in the front, /node/ handled by local Node.

STEP 1: Add this to some .conf file:
ProxyPass /node/ http://localhost:8000/
(e.g. save it in some my_proxy_forward_node_to_8000.conf, then a2enconf it) More notes for this to work: (from ProxyPass documentation)
  • Ensure the mods proxy and proxy_http are enabled (a2enmod).
  • If the first argument ends with a trailing /, the second argument should also end with a trailing /, and vice versa. Otherwise, the resulting requests to the backend may miss some needed slashes and do not deliver the expected results.
  • The ProxyRequests directive should usually be set off when using ProxyPass.
STEP 2: Then run Node locally on port 8000:
var http = require('http');
http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Hello Apache!\n');
}).listen(8000, '127.0.0.1');

Option 2. TrafficServer in the front, Apache and Node behind it.

STEP 0: Install TrafficServer. (Note: As of 4/2017, this installed version 5.3.x from the repositories, though the real latest version is 7.2.x)
apt-get install trafficserver
IMPORTANT: Edit records.config and add this line (I added in the beginning; I don’t understand why this isn’t the default configuration). If you don’t specify such user_id, it will not agree to run itself at all, because it doesn’t want to run as root.
CONFIG proxy.config.admin.user_id STRING trafficserver
STEP 1: Enable Reverse Proxying (pass incoming traffic to e.g. another port in localhost, preserving “Host:” headers etc.). Edit records.config: (the underlined value here is the only change needed for the default value, as of 4/2017)
CONFIG proxy.config.http.cache.http INT 1
CONFIG proxy.config.reverse_proxy.enabled INT 1
CONFIG proxy.config.url_remap.remap_required INT 1
CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
CONFIG proxy.config.http.server_ports STRING 8080
Edit remap.config:
regex_map http://(.*):8080/ http://localhost:80/
Finally, reread the config files (traffic_line --reread_config or equivalently -x). IMPORTANT: The 8080 and 80 above (in both records.config and remap.config) should be switched if you really want TrafficServer in the front… STEP 2: Log files, monitoring, security: https://docs.trafficserver.apache.org/en/5.3.x/admin/working-log-files.en.html https://docs.trafficserver.apache.org/en/5.3.x/admin/monitoring-traffic.en.html https://docs.trafficserver.apache.org/en/5.3.x/admin/security-options.en.html STEP 3: If you want to disable cache completely: (didn’t check this) Edit record.config:
CONFIG proxy.config.http.cache.http INT 0

(BONUS) STEP: Cache configuration: storage.config. E.g. its default contents were simply a one-liner /var/cache/trafficserver 256M. Changes require restart of trafficserver. It is recommended to use raw devices; see documentation. E.g. how to cache everything (e.g. if you’re serving only static files): By default, Traffic Server will cache an HTTP response only if it contains a Cache-Control or Expires header explicitly specifying how long the item should be stored in the cache. To disable that need for those required headers:
sudo traffic_line --set_var proxy.config.http.cache.required_headers --value 0
sudo traffic_line --reread_config
(BONUS) STEP: Use a tool called Cache Inspector:
sudo traffic_line --set_var proxy.config.http_ui_enabled --value 1
Edit remap.config, add this line at the top of the file:
map http://your_server_ip:8080/inspect http://{cache}
Then restart the trafficserver service.

WordPress Twenty Seventeen modifications

The correct way is to work with Child Themes instead of the following hacks. To disable WordPress editor from inserting annoying <p> for new lines, add this in functions.php:
remove_filter( 'the_content', 'wpautop' );
remove_filter( 'the_excerpt', 'wpautop' );
To remove the “Proudly powered by WordPress”, comment out the line that fetches this template part in footer.php:
//get_template_part( 'template-parts/footer/site', 'info' );
To make post’s primary content slightly wider at the expense of the sidebar (58%:36% => 65%:31%), simply add more CSS in Appearance => Customize: (or alternatively modify directly these rows in style.css)
@media screen and (min-width: 48em) {
.has-sidebar:not(.error404) #primary { width: 65%; }
}
@media screen and (min-width: 48em) {
.has-sidebar #secondary { width: 31%; }
}

File does not exist: /etc/apache2/htdocs (in your apache error.log file)

The problem

/var/log/apache2/error.log has these annoying message every 5 minutes: [Wed Mar 30 18:12:34 2016] [error] [client 127.0.0.1] File does not exist: /etc/apache2/htdocs [Wed Mar 30 18:17:35 2016] [error] [client 127.0.0.1] File does not exist: /etc/apache2/htdocs [Wed Mar 30 18:22:36 2016] [error] [client 127.0.0.1] File does not exist: /etc/apache2/htdocs [Wed Mar 30 18:27:36 2016] [error] [client 127.0.0.1] File does not exist: /etc/apache2/htdocs [Wed Mar 30 18:32:37 2016] [error] [client 127.0.0.1] File does not exist: /etc/apache2/htdocs [Wed Mar 30 18:37:37 2016] [error] [client 127.0.0.1] File does not exist: /etc/apache2/htdocs [Wed Mar 30 18:42:38 2016] [error] [client 127.0.0.1] File does not exist: /etc/apache2/htdocs [Wed Mar 30 18:47:39 2016] [error] [client 127.0.0.1] File does not exist: /etc/apache2/htdocs

The solution

Create the file /etc/apache2/conf.d/default-documentroot and have it contain this single line: DocumentRoot /var/www Then restart the apache2 service: service apache2 restart

Gulp

First note: We’ll eventually be using some of these packages: (also note the --save vs. --save-dev)
npm install --save-dev gulp gulp-nodemon gulp-htmlmin gulp-cssnano gulp-uglify gulp-buster browserify vinyl-source-stream gulp-util babelify babel-preset-es2015 babel-preset-react
npm install --save react react-dom
Ok, hi. Let’s start with gulp‘s .task() simple syntax:
// (in gulpfile.js)
gulp.task('temp', [], function(){
  console.log('hello world');
}
Just copy the above into the file gulpfile.js at your root directory — this is the only file gulp needs in order to run. Then run %> gulp temp and get:
hello world
And here’s how we can simply bundle together a few files, using browserify:
// (in gulpfile.js)
gulp.task('temp', [], function() {
  browserify({
    entries: 'file1.js' // or array, e.g. ['file1.js','file2.js']
  })
  .add('another2.js') // Can also add like this
  .add('another3.js') // And more...
  .add(['another4.js', 'another5.js']) // Or use an array...

  .bundle() // Finally, let's bundle!

  .pipe(source('bundle.js')) // And store result in bundle.js
  .pipe(gulp.dest('./')); // In this directory
Ok, so we have the following simple usage:
// (in gulpfile.js)
gulp.task('temp', [], function() {
  browserify({
    entries: 'file1.js'
  })
  .bundle() // Bundle!
  .pipe(source('bundle.js')) // Store result in bundle.js
  .pipe(gulp.dest('./')); // In this directory
Suppose file1.js contains:
// file1.js
var file1var = 123;
console.log('in file1.js');
Then running %> gulp temp will overwrite bundle.js to contain:
(function e(t,n,r){function s(o,u){if(!n[o]){if(!t[o]){var a=typeof require=="function"&&require;if(!u&&a)return a(o,!0);if(i)return i(o,!0);var f=new Error("Cannot find module '"+o+"'");throw f.code="MODULE_NOT_FOUND",f}var l=n[o]={exports:{}};t[o][0].call(l.exports,function(e){var n=t[o][1][e];return s(n?n:e)},l,l.exports,e,t,n,r)}return n[o].exports}var i=typeof require=="function"&&require;for(var o=0;o<r.length;o++)s(r[o]);return s})({1:[function(require,module,exports){
// file1.js
var file1var = 123;
console.log('in file1.js');

},{}]},{},[1]);
There you have it. The basic usage we’ve been looking for: We can bundle together multiple JS files into one, so that we could include it in from our index.html using some script tag, for example. In our modest case, though, with no index.html and just dummy code, we could run %> node bundle.js to get this output:
in file1.js

It’s time we appreciate what we already have in our hands: What bundle really does is “Bundle the files and their dependencies into a single javascript file.” Let’s see that in action, shall we? Create a file2.js:
var file2var = 456;
console.log('in file2.js');

module.exports = {
  f : function() {
    console.log('My cool function from file2.js');
  },
  c : 789
}
And add an import command to file1.js, so its updated content is:
import File2 from './file2';

// file1.js
var file1var = 123;
console.log('in file1.js');
Running %> gulp temp will FAIL, with some error like the one below, complaining about our ES2015 syntax… (namely our newly added import command in file1.js)
events.js:141
      throw er; // Unhandled 'error' event
      ^
SyntaxError: 'import' and 'export' may appear only with 'sourceType: module'
So to allow ourselves to write in ES2015 and transpile it to ordinary JS, we use babelify! Update our gulp file as follows:
// (in gulpfile.js)
gulp.task('temp', [], function() {
  browserify({
    entries: 'file1.js'
  })
  .transform("babelify", {presets: ["es2015"]}) // <= ADD THIS ROW
  .bundle() // Bundle!
  .pipe(source('bundle.js')) // Store result in bundle.js
  .pipe(gulp.dest('./')); // In this directory
And voila, running %> gulp temp overwrites bundle.js into the following:
(function e(t,n,r){function s(o,u){if(!n[o]){if(!t[o]){var a=typeof require=="function"&&require;if(!u&&a)return a(o,!0);if(i)return i(o,!0);var f=new Error("Cannot find module '"+o+"'");throw f.code="MODULE_NOT_FOUND",f}var l=n[o]={exports:{}};t[o][0].call(l.exports,function(e){var n=t[o][1][e];return s(n?n:e)},l,l.exports,e,t,n,r)}return n[o].exports}var i=typeof require=="function"&&require;for(var o=0;o<r.length;o++)s(r[o]);return s})({1:[function(require,module,exports){
'use strict';

var _file = require('./file2');

var _file2 = _interopRequireDefault(_file);

function _interopRequireDefault(obj) { return obj && obj.__esModule ? obj : { default: obj }; }

// file1.js
var file1var = 123;
console.log('in file1.js');

},{"./file2":2}],2:[function(require,module,exports){
'use strict';

var file2var = 456;
console.log('in file2.js');

module.exports = {
  f: function f() {
    console.log('My cool function from file2.js');
  },
  c: 789
};

},{}]},{},[1]);
Cool! So even though we merely asked for file1.js to be bundled, browserify realized that it depends on file2.js and included it in the bundle! Awesome! The import command in file1.js first "sources" the contents of file2.js. To illustrate, try running %> node bundle.js, you'll get:
in file2.js
in file1.js

Finally, a more closer-to-reality usage is to have file1.js do something with the class / object that we imported from file2.js. For example, suppose file1.js is:
import File2 from './file2';

// file1.js
var file1var = 123;
console.log('in file1.js');

console.log(File2);
Run %> gulp temp and then %> node bundle.js and we get:
in file2.js
in file1.js
{ f: [Function: f], c: 789 }

Read more here on the import command and ways to selectively import only some of the module's members (and even give them shorter aliases if necessary). Read here on browserify. In particular, about require and external!

Input type=number

<input type="number" step="any"/> Will allow any decimal point. If the current value of the input is “123.456”, for example, then clicking the UP arrow will increase it to “124.456”. Without the ‘step=”any”‘, it would’ve increased to “124”. Note: The ‘step’ attribute, together with ‘min’ and ‘max’, control which values are considered valid.

jQuery Cheatsheet

Traversal

$('p').find('span') Finds all span elements that are descendants of some ‘p’ elements (not necessarily immediate children). Compare to .children(), that travels only a single level down the DOM tree.
$('p').find('span')
or: $('span', $('p'))
Finds all span elements that are descendants of some ‘p’ elements (not necessarily immediate children). Compare to .children(), that travels only a single level down the DOM tree.

Manipulation

$(this).data('k',123) Use $(this).data('k') to get the associated value (123 in this case), $(this).data() to get all the key-value map, and $(this).removeData('k') to remove a data entry.
$('<div/>',{'class':'c1 c2'}) As of jQuery 1.4, this creates an element on the fly with specified attributes, e.g. <div class="c1 c2"></div> in this case. Note: Usually you’d like to append() or appendTo() this new element. But if you want its string representation, use the element.outerHTML attribute, i.e. $(...)[0].outerHTML.

Which PayPal product should I choose

Many products, many APIs
With their inconsistent information (and user experience) all over their dozens of different domains, PayPal miserably fails to deliver a clear message to developers. Here is a nice attempt (by PayPal) to consolidate some typical use cases so you can decide better which product is suitable for you, but you might get lost without some basic understanding of terms like “Express Checkout”, “Adaptive Payments”, “Classic APIs” etc. And so we try to shed some light on the general usage of PayPal’s products and APIs. Leave a comment and we’ll consider elaborating on other stuff as well. This document is still in the process of writing. Feel free to comment with feedback and suggestions.
PayPal NVP (Name-Value Pair) API
PayPal exist for many years, so they host a few families of APIs you can use. The newest are their REST APIs. In this section however we focus on the NVP API, which is considered as one of their “Classic APIs”. The “Name-Value” simply means that we’re passing information using the format “NAME=value”, for example FIRSTNAME=Robert&MIDDLENAME=Herbert&LASTNAME=Moore. Our page will send such a formatted request to the PayPal API Server, which will return a response: There’s a specific way to build the request. For example, you need to specify the METHOD (the required API operation to perform), and you need to provide some API Credentials: Either a signature or a certificate (but not both). For example, the “signature” option is merely some “USER=...&PWD=...&SIGNATURE=...” (together with the METHOD=... as well as other request-specific details. The response is also formatted in a specified way, and both response and request should be UTF-8-encoded. But it may help to first understand the types of requests (“API operation”s) you can make, so read on! (and later feel free to dwell into the boring details in the original documentation) Note: Some of the features, like Express Checkout, require your process to perform multiple operations in order to complete. For example, first call SetExpressCheckout and then DoExpressCheckoutPayment, as illustrated here:
Classic APIs
The API operations are divided in the documentation into 6 categories:
  • Adaptive Accounts: Create PayPal accounts and handle account management. Example requests: CreateAccount, AddPaymentCard, …
  • Adaptive Payments: Manage payments, payment preapprovals, and refunds. Example requests: Execute Payment, Pay, Refund, SetPaymentOptions, …
  • Button Manager: Manage your buttons! BMCreateButton, BMGetButtonDetails, … See also the image reference here.
  • Invoicing: CreateInvoice, SendInvoice, …
  • Merchant APIs: Some useful operations here! E.g. DoDirectPayment, DoExpressCheckoutPayment, CreateRecurringPaymentsProfile, GetExpressCheckoutDetails, GetRecurringPaymentsProfileDetails, SetExpressCheckout, UpdateRecurringPaymentsProfile, …
  • Permissions: GetPermissions, RequestPermissions, … Handle permissions associated with an access token, e.g. to execute operations on behalf of the buyer.
Express Checkout
The typical use case: When you want to allow one-time payment (as opposed to recurring payments), for example to support shopping carts on your site. The buyer clicks a button at your site, redirected to PayPal to fill in some payment details and confirm, and then redirected back to your site. You must have a PayPal merchant account to allow this. Below is a summary of the getting started guide.
  • The typical checkout flow, without Express Checkout, is the following:
    • The payor (buyer) clicks the checkout button in his shopping cart page,
    • Provide shipping info,
    • Select payment method,
    • Review summary for approval,
    • Sees a confirmation of the transaction.
  • The “Express Checkout” way skips the “shipping info” and “payment method” steps. The image below neatly summarizes the process together with the 3 API calls your applications needs to make:
The relevant API for working with Express Checkout is here. A good and detailed documentation about actually implementing it is here.
Payflow Gateway
For the more serious applications that want full control on the process: To host the checkout pages, manage security for sales and authorizations. You need an “Internet Merchant Account” for that (which is a different thing from being simply a “merchant”, i.e. there’s a special application form you have to fill and it gets processed within a few days). Quoting the documentation:
Payflow is secure, open payment gateway. It is available in two versions: Payflow Link, which requires the use of hosted payment pages, and Payflow Pro, which gives merchants total control over the payment process. Payflow allows merchants to choose any Internet Merchant Account to accept debit or credit card payments and connect to any major processor. It also lets merchants accept PayPal and Bill Me Later payments.
PayPal Payments Pro (Direct Integration), Website Payments Pro
Enables merchants and developers to accept credit cards, debit cards, and PayPal payments directly on their website. PayPal Payments Pro also includes Express Checkout and Virtual Terminal. Add-on services like Recurring Billing are also available.
Recurring Payments
https://developer.paypal.com/docs/classic/paypal-payments-pro/integration-guide/WPRecurringPayments/ recurring payment profile: Every time someone is charged in a recurring manner (subscription), in PayPal terms this means she has a recurring payment profile. This “profile” instance includes all the payments details and is maintained at PayPal’s side.