Cloud Computing with FloydHub

2017-09-18 本文已影响0人 sterio

Create an account

Install `floyd-cli` on computer with dependency of python.

pip install -U floyd-cli

Installing...

steriowang:~ sterio$ pip install -U floyd-cli
Collecting floyd-cli
  Downloading floyd-cli-0.10.9.tar.gz
Requirement already up-to-date: click>=6.7 in ./anaconda3/lib/python3.6/site-packages (from floyd-cli)
Collecting clint>=0.5.1 (from floyd-cli)
  Downloading clint-0.5.1.tar.gz
Collecting requests>=2.12.4 (from floyd-cli)
  Downloading requests-2.18.4-py2.py3-none-any.whl (88kB)
    100% |████████████████████████████████| 92kB 31kB/s 
Collecting requests-toolbelt>=0.7.1 (from floyd-cli)
  Downloading requests_toolbelt-0.8.0-py2.py3-none-any.whl (54kB)
    100% |████████████████████████████████| 61kB 36kB/s 
Collecting marshmallow>=2.11.1 (from floyd-cli)
  Downloading marshmallow-2.13.6-py2.py3-none-any.whl (45kB)
    100% |████████████████████████████████| 51kB 85kB/s 
Requirement already up-to-date: pytz>=2016.10 in ./anaconda3/lib/python3.6/site-packages (from floyd-cli)
Collecting shortuuid>=0.4.3 (from floyd-cli)
  Downloading shortuuid-0.5.0.tar.gz
Collecting tabulate>=0.7.7 (from floyd-cli)
  Downloading tabulate-0.7.7-py2.py3-none-any.whl
Collecting pathlib2>=2.2.1 (from floyd-cli)
  Downloading pathlib2-2.3.0-py2.py3-none-any.whl
Collecting args (from clint>=0.5.1->floyd-cli)
  Downloading args-0.1.0.tar.gz
Collecting chardet<3.1.0,>=3.0.2 (from requests>=2.12.4->floyd-cli)
  Downloading chardet-3.0.4-py2.py3-none-any.whl (133kB)
    100% |████████████████████████████████| 143kB 58kB/s 
Collecting urllib3<1.23,>=1.21.1 (from requests>=2.12.4->floyd-cli)
  Downloading urllib3-1.22-py2.py3-none-any.whl (132kB)
    100% |████████████████████████████████| 133kB 71kB/s 
Collecting idna<2.7,>=2.5 (from requests>=2.12.4->floyd-cli)
  Downloading idna-2.6-py2.py3-none-any.whl (56kB)
    100% |████████████████████████████████| 61kB 93kB/s 
Collecting certifi>=2017.4.17 (from requests>=2.12.4->floyd-cli)
  Downloading certifi-2017.7.27.1-py2.py3-none-any.whl (349kB)
    100% |████████████████████████████████| 358kB 172kB/s 
Collecting six (from pathlib2>=2.2.1->floyd-cli)
  Downloading six-1.11.0-py2.py3-none-any.whl
Building wheels for collected packages: floyd-cli, clint, shortuuid, args
  Running setup.py bdist_wheel for floyd-cli ... done
  Stored in directory: /Users/sterio/Library/Caches/pip/wheels/27/5e/b8/75d2e8df1a67c9a72a90d133bd565a86d8a48af0b0527215ed
  Running setup.py bdist_wheel for clint ... done
  Stored in directory: /Users/sterio/Library/Caches/pip/wheels/aa/65/4a/a0bceb0a6bee518b1dace4c803f24c4ac8febe55bdca0f14ed
  Running setup.py bdist_wheel for shortuuid ... done
  Stored in directory: /Users/sterio/Library/Caches/pip/wheels/0b/50/5e/fae6e070b59ef891e16b518376682f9b084491c1832b99cc34
  Running setup.py bdist_wheel for args ... done
  Stored in directory: /Users/sterio/Library/Caches/pip/wheels/0a/e0/66/35419f9f9ee41099a3631f0f308b9604bd1ffeabc066d69070
Successfully built floyd-cli clint shortuuid args
Installing collected packages: args, clint, chardet, urllib3, idna, certifi, requests, requests-toolbelt, marshmallow, shortuuid, tabulate, six, pathlib2, floyd-cli
  Found existing installation: chardet 2.3.0
    Uninstalling chardet-2.3.0:
      Successfully uninstalled chardet-2.3.0
  Found existing installation: idna 2.2
    Uninstalling idna-2.2:
      Successfully uninstalled idna-2.2
  Found existing installation: requests 2.13.0
    Uninstalling requests-2.13.0:
      Successfully uninstalled requests-2.13.0
  Found existing installation: six 1.10.0
    DEPRECATION: Uninstalling a distutils installed project (six) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
    Uninstalling six-1.10.0:
      Successfully uninstalled six-1.10.0
  Found existing installation: pathlib2 2.2.0
    Uninstalling pathlib2-2.2.0:
      Successfully uninstalled pathlib2-2.2.0
Successfully installed args-0.1.0 certifi-2017.7.27.1 chardet-3.0.4 clint-0.5.1 floyd-cli-0.10.9 idna-2.6 marshmallow-2.13.6 pathlib2-2.3.0 requests-2.18.4 requests-toolbelt-0.8.0 shortuuid-0.5.0 six-1.11.0 tabulate-0.7.7 urllib3-1.22

Login in to FloydHub through `floyd-cli`

steriowang:~ sterio$ floyd login
Authentication token page will now open in your browser. Continue? [Y/n]: y
Please copy and paste the authentication token.
This is an invisible field. Paste token and press ENTER: 
Login Successful

CLI authentication

Get code from Github with `git clone` + `github repository url`

steriowang:~ sterio$ git clone https://github.com/sterio-wang/Udacity-dlnd-image-classification-with-CIFAR-10-dataset
Cloning into 'Udacity-dlnd-image-classification-with-CIFAR-10-dataset'...
remote: Counting objects: 13, done.
remote: Total 13 (delta 0), reused 0 (delta 0), pack-reused 13
Unpacking objects: 100% (13/13), done.

cd cd + project nameand list all files with ls command in that repository

steriowang:~ sterio$ cd Udacity-dlnd-image-classification-with-CIFAR-10-dataset
steriowang:Udacity-dlnd-image-classification-with-CIFAR-10-dataset sterio$ ls
README.md           helper.py
dlnd_image_classification.html  problem_unittests.py
dlnd_image_classification.ipynb
steriowang:Udacity-dlnd-image-classification-with-CIFAR-10-dataset sterio$

It contains 5 files:

README.md
helper.py
dlnd_image_classification.html
problem_unittests.py
dlnd_image_classification.ipynb

Initialize selected project

Take jupyter notebook mode as example. Firstly initialize selected project byfloyd init + project name. Then run

$ floyd run --mode jupyter --gpu --env tensorflow

It will execute 4 things behind:

steriowang:Udacity-dlnd-image-classification-with-CIFAR-10-dataset sterio$ floyd run --mode jupyter --gpu --env tensorflow
Creating project run. Total upload size: 536.9KiB
Syncing code ...
[================================] 551280/551280 - 00:00:04

JOB NAME
-------------------------------------------------------------------------
sterio/projects/udacity-dlnd-image-classification-with-cifar-10-dataset/1

Setting up your instance and waiting for Jupyter notebook to become available ......

Path to jupyter notebook: https://www.floydhub.com/notebooks/vhiz2KgzPM8snPvi9MRqB2

Sync your local code to FloydHub's server
Provision a GPU instance on the cloud (if you want CPU, drop the --gpu flag)floyd run --mode jupyter --env tensorflow
Set up an deep learning environment with TensorFlow installed (because --env tensorflow)
Start a Jupyter server on the cloud, and open the url in your browser as:

Floyd Jupyter Server

After jupyter server page pops up then you can open and run the ipynb cell by cell just like running locally. It's recommended to use GPU for efficiency. Here is the output of my project: https://www.floydhub.com/notebooks/T4wUwkxFLHLebC4YkxUa8i/notebooks/dlnd_image_classification.ipynb

Supplementary

Monitor project status by command floyd status

steriowang:Udacity-dlnd-image-classification-with-CIFAR-10-dataset sterio$ floyd status
JOB NAME                                                                   CREATED        STATUS      DURATION(s)  INSTANCE    DESCRIPTION
-------------------------------------------------------------------------  -------------  --------  -------------  ----------  -------------
sterio/projects/udacity-dlnd-image-classification-with-cifar-10-dataset/5  8 minutes ago  running               0  g1

Delete a job by command floyd delete job name or Web UI

steriowang:Udacity-dlnd-image-classification-with-CIFAR-10-dataset sterio$ floyd delete sterio/projects/udacity-dlnd-image-classification-with-cifar-10-dataset/5
Delete Run: sterio/projects/udacity-dlnd-image-classification-with-cifar-10-dataset/5? [y/N]: y
Job HPLamsSwAyj7fNgh6YXdtH: Deleted
steriowang:Udacity-dlnd-image-classification-with-CIFAR-10-dataset sterio$

Check computing and storage usage here: https://www.floydhub.com/settings/usage
Encrypt private project: http://jupyter-notebook.readthedocs.io/en/latest/public_server.html#securing-a-notebook-server

Cloud Computing with FloydHub

Create an account

Install `floyd-cli` on computer with dependency of python.

Login in to FloydHub through `floyd-cli`

Get code from Github with `git clone` + `github repository url`

Initialize selected project

Supplementary

猜你喜欢

热点阅读

Cloud Computing with FloydHub

Create an account

Install floyd-cli on computer with dependency of python.

Login in to FloydHub through floyd-cli

Get code from Github with git clone + github repository url

Initialize selected project

Supplementary

猜你喜欢

热点阅读

Install `floyd-cli` on computer with dependency of python.

Login in to FloydHub through `floyd-cli`

Get code from Github with `git clone` + `github repository url`