Docker in Windows, allocate more memory

Some installation / cheatsheet

Using Docker for Windows, Windows 10. 17.03.0-ce-win1 (10296). Channel: stable. 94675c5.

(Tip: Also install Kitematic, then you can right-click the running Docker icon and from there launch Kitematic for a nice GUI to see your containers.)

Basic commands:

# Create a tensorflow container:
docker run -it -p 8888:8888 -p 6006:6006 --name my_tensor_flow -v C:/Data/Docker:/data -v C:/Dev/python:/devpython tensorflow/tensorflow
# Run it in the future:
docker start -ai my_tensor_flow
# To connect to it with bash: (note: /notebooks is where the notebooks are kept, and we can already access the host's C:/Data/Docker in /data)
docker exec -it my_tensor_flow bash
# Then from the bash can run: tensorboard --logdir=/data/tensorboard/5/

Apart from the nice GUI, there are some useful commands from the command line:

docker stats [--all]
docker ps [--all]
docker container list [--all]

Allocating more memory

For my running container, docker stats showed (in “MEM USAGE / LIMIT”) that it was bounded by 1.934 GiB. Indeed, “Hyper-V Manager” showed “MobyLinuxVM” machine has only 2GB.

To allocate more memory: Docker’s settings, Advanced, set Docker’s memory e.g. to 8448MB. Docker will restart. Verify that “Hyper-V Manager” now shows “MobyLinuxVM” has assigned memory of 8448 MB. Run your previous container docker start -ai container-name, and now docker stats will show a memory bound of 8.003 GiB. Success.

Note: Running some

docker run -it -p ... --name ... --memory-swap -1 --memory 8g -v ...
will not work if your “MobyLinuxVM” doesn’t have enough assigned memory.

Running Apache and Node on the same server

Option 1. Apache in the front, /node/ handled by local Node.

STEP 1: Add this to some .conf file:
ProxyPass /node/ http://localhost:8000/
(e.g. save it in some my_proxy_forward_node_to_8000.conf, then a2enconf it) More notes for this to work: (from ProxyPass documentation)
  • Ensure the mods proxy and proxy_http are enabled (a2enmod).
  • If the first argument ends with a trailing /, the second argument should also end with a trailing /, and vice versa. Otherwise, the resulting requests to the backend may miss some needed slashes and do not deliver the expected results.
  • The ProxyRequests directive should usually be set off when using ProxyPass.
STEP 2: Then run Node locally on port 8000:
var http = require('http');
http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Hello Apache!\n');
}).listen(8000, '127.0.0.1');

Option 2. TrafficServer in the front, Apache and Node behind it.

STEP 0: Install TrafficServer. (Note: As of 4/2017, this installed version 5.3.x from the repositories, though the real latest version is 7.2.x)
apt-get install trafficserver
IMPORTANT: Edit records.config and add this line (I added in the beginning; I don’t understand why this isn’t the default configuration). If you don’t specify such user_id, it will not agree to run itself at all, because it doesn’t want to run as root.
CONFIG proxy.config.admin.user_id STRING trafficserver
STEP 1: Enable Reverse Proxying (pass incoming traffic to e.g. another port in localhost, preserving “Host:” headers etc.). Edit records.config: (the underlined value here is the only change needed for the default value, as of 4/2017)
CONFIG proxy.config.http.cache.http INT 1
CONFIG proxy.config.reverse_proxy.enabled INT 1
CONFIG proxy.config.url_remap.remap_required INT 1
CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
CONFIG proxy.config.http.server_ports STRING 8080
Edit remap.config:
regex_map http://(.*):8080/ http://localhost:80/
Finally, reread the config files (traffic_line --reread_config or equivalently -x). IMPORTANT: The 8080 and 80 above (in both records.config and remap.config) should be switched if you really want TrafficServer in the front… STEP 2: Log files, monitoring, security: https://docs.trafficserver.apache.org/en/5.3.x/admin/working-log-files.en.html https://docs.trafficserver.apache.org/en/5.3.x/admin/monitoring-traffic.en.html https://docs.trafficserver.apache.org/en/5.3.x/admin/security-options.en.html STEP 3: If you want to disable cache completely: (didn’t check this) Edit record.config:
CONFIG proxy.config.http.cache.http INT 0

(BONUS) STEP: Cache configuration: storage.config. E.g. its default contents were simply a one-liner /var/cache/trafficserver 256M. Changes require restart of trafficserver. It is recommended to use raw devices; see documentation. E.g. how to cache everything (e.g. if you’re serving only static files): By default, Traffic Server will cache an HTTP response only if it contains a Cache-Control or Expires header explicitly specifying how long the item should be stored in the cache. To disable that need for those required headers:
sudo traffic_line --set_var proxy.config.http.cache.required_headers --value 0
sudo traffic_line --reread_config
(BONUS) STEP: Use a tool called Cache Inspector:
sudo traffic_line --set_var proxy.config.http_ui_enabled --value 1
Edit remap.config, add this line at the top of the file:
map http://your_server_ip:8080/inspect http://{cache}
Then restart the trafficserver service.

File does not exist: /etc/apache2/htdocs (in your apache error.log file)

The problem

/var/log/apache2/error.log has these annoying message every 5 minutes: [Wed Mar 30 18:12:34 2016] [error] [client 127.0.0.1] File does not exist: /etc/apache2/htdocs [Wed Mar 30 18:17:35 2016] [error] [client 127.0.0.1] File does not exist: /etc/apache2/htdocs [Wed Mar 30 18:22:36 2016] [error] [client 127.0.0.1] File does not exist: /etc/apache2/htdocs [Wed Mar 30 18:27:36 2016] [error] [client 127.0.0.1] File does not exist: /etc/apache2/htdocs [Wed Mar 30 18:32:37 2016] [error] [client 127.0.0.1] File does not exist: /etc/apache2/htdocs [Wed Mar 30 18:37:37 2016] [error] [client 127.0.0.1] File does not exist: /etc/apache2/htdocs [Wed Mar 30 18:42:38 2016] [error] [client 127.0.0.1] File does not exist: /etc/apache2/htdocs [Wed Mar 30 18:47:39 2016] [error] [client 127.0.0.1] File does not exist: /etc/apache2/htdocs

The solution

Create the file /etc/apache2/conf.d/default-documentroot and have it contain this single line: DocumentRoot /var/www Then restart the apache2 service: service apache2 restart

Git Quick Tutorial, Cheat Sheet

The following is meant to be a quick cheatsheet to recap some Git commands.

Basic flow

An example basic flow of commands:
git initto create an empty Git repository in a .git folder.
git statusto see which files are not tracked, changes, conflicts etc.
git add .to add all files to the staging area (a.k.a. “index”).
(if you modify a file now and want to commit those changes, don’t forget to git add it again before committing)
git reset file.txtto undo that last add command for file.txt, so now it’s not tracked.
git commit -m "my commit remark" creates a new commit object with the staged changes in the Git repository.

Deleting files

Create some file and commit it: touch deleteme.txt git add . git commit -m "added deleteme.txt" Will commit the deleteme.txt file
Try to delete (will FAIL): rm deleteme.txt git add . git commit -m "will fail" Will get a no changes to commit message, and the deleteme.txt file will still be kept in the repository…
How to delete (alternative 1): git add -A . git commit -m "success" The -A (or equivalently --all) compares changes in the working tree as well as the index, so the file deleteme.txt will now be deleted in the repository as well.
How to delete (alternaive 2): git commit -a -m "success" The -a (or equivalently --all) tells the commit command to automatically stage files that have been modified and deleted, so the file deleteme.txt will now be deleted in the repository as well.
How to delete (alternaive 3): Instead of the rm deleteme.txt above, could instead: git rm deleteme.txt git commit -m "success" The rm command removes the file and records its deletion in Git, so after the commit command the deleteme.txt will be deleted in the repository as well.

Differences

git diff shows the changes in the working tree not yet staged for the next commit. git diff --cached shows the changes between the index and your last commit; what you would be committing if you run “git commit” without “-a” option. In any case, it ignores any non-tracked files.
echo "some text" > file.txt git add file.txt git commit -m "remark1" Suppose we add a file.txt and commit it.
Now modify it: echo "more text" >> file.txt Modified, but still didn’t add it to staging, i.e. we didn’t run: git add file.txt
git diff will show us the differences we have in that tracked file! Example diff output: diff --git a/file.txt b/file.txt index 477f637..f8217a9 100644 --- a/file.txt +++ b/file.txt @@ -1 +1,2 @@ "some text" +"more text"
But after adding the changes there will be no diff: git add file.txt git diff So the git diff will not print anything now. If you want to see the changes between the index and the last commit, use git diff --cached (will produce the same diff output as above)

Remotes and more

git remote List the existing remotes whose branches you track.
git remote -v A bit more verbose, show remote url after name.
– Add a remote repository: Use ‘git remote add’ with some remote name (usually “origin” is the name of the main remote) and repository URL: git remote add origin https://github.com/try-git/try_git.git – Now can push local changes remotely to our ‘origin’ repo (on GitHub): The name of our remote is ‘origin’. The default branch name is ‘master’. git push -u origin master (the -u tells Git to remember the parameters, so that next time we can simply run git push and Git will know what to do) – Suppose others worked on our project and committed stuff. We can check for changes on our GitHub repository and pull down any new changes by running: git pull origin master – If we want the diff of our most recent commit, which we can refer to using the HEAD pointer: git diff HEAD – Another great use for diff is looking at changes within files that have already been staged/cached (added, but not committed yet): git diff --cached (--cached is same as --staged) – Other example for git diff: git diff v2.5:Makefile HEAD:Makefile.in – To unstage files: git reset octofamily/octodog.txt – To revert the working tree to the state of some branch or specific paths: git checkout master (checks out the master branch (switches branch)) git checkout master~2 Makefile (takes a file out of another commit. Reverts the Makefile to two revisions back) – Now suppose we accidentally ran ‘rm -f hello.c‘. Then we could restore it from the index: git checkout hello.c (but in case we have a branch called hello.c, this will be understood as reverting to that branch!!! So better to use the following:) git checkout -- hello.c – Branching out: git branch my_branch – To see the branches: (with “*” at the current branch) git branch – To switch to another branch: (also discussed above) git checkout my_branch – In that branch, suppose we remove some files from both the disk and from the staging/index: git rm '*.txt' – And commit the change: git commit -m "cleaned up stuff" – Now we want to copy (merge) changes from my_branch back into the master branch: git checkout master git merge my_branch – To delete a branch: git branch -d my_branch – Finally, can again push it to the remote repo: git push
git log Shows the recent commits etc.

Other resources