I host my personal websites on DreamHost, but it was an ordeal setting up Django using Python 3 in a virtual Python environment. DreamHost's documentation don't really cover this particular permutation. So here's a quick how-to.
Firstly, a quick explainer on the end-goal of how web requests will be routed through the server.
DreamHost uses Apache to handle HTTP requests. We want to use Python to handle some requests, but use Apache to serve the static files (HTML, CSS, etc), as using Python for a general purpose server would not be performant enough. DreamHost recommends and prefers the use of Passenger to handle web requests for Python. Passenger was designed for Ruby, but can be used for Python as well.
Your user's web browser makes requests to your DreamHost's server. The server routes the requests to Passenger, which in turn routes them to your WSGI handler - which will be a Django Project in our case. If the WSGI handler doesn't handle the request, it gets routed back to the Apache to handle. For example requests for a static asset (i.e. an HTML or CSS file) would go to Passenger, not be handled, and then fall back to be handled by the web server. When Passenger is invoked, it will start up an instance of your Python code in a stand alone process for handling requests. This process will be kept alive for some time, and any further requests that come in for the next wee while will be serviced by this process. So your process may be re-used for multiple requests, but you can't guarantee that it will, as Passenger will shutdown your process after some time period of inactivity.
Passenger will by default start up the Python WSGI handler using the system default Python, which at the time of writing on my DreamHost server was still Python 2. In order to get Python 3 inside our virtualenv, we'll need to re-invoke with our virtualenv's Python 3 interpreter.
You don't want your code to be web accessible. So in your home directory you'll create a directory for your code, a directory for your static assets (HTML/CSS files) that Apache uses as the root, and a directory for your Python 3 virtualenv and pip dependencies.
For a database, I simply used SqlLite, with the database file stored inside the ~/app directory. That was easy to setup and is fine for a simple learning project. I never figured out how to get Python 3 to talk to DreamHost's MySQL server; IIRC the native libraries required for Python 3 to talk to MySQL weren't installed on the server I'm on. I expect if you emailed DreamHost's support team they could install the packages for you to enable Python 3 to talk to their MySQL server. That said, for small workloads, I've found SqlLite to perform better than a full-blown MySQL server.
With that all said, let's dive into the specifics of setting this up.
From the DreamHost control panel, create a new domain website under Domains > Manage Domains. Make sure you turn on the "Enable Passenger" option, and follow the advice about the web directory ending in "public". Make a note of the username, password and server that you were assigned. I'm going to assume your Web Directory is www/public in the examples below.
I'll use the convention that anything that could vary based on your setup that you may need to change in the commands/config below will be in bold. For example the username that DreamHost assigns to your new domain, or the Web Directory.
Still in the Control Panel, under FTP & SSH > Manage Users, select your new domain's user, and enable SSH.
Under Domains > SSL/TLS Certificates, select your new domain, and add a free Lets Encrypt certificate. This will mean you'll be able to use HTTPS on your site.
For convenience, I would then follow DreamHost's instructions to enable passwordless login via SSH keys. For me this was as simple as:
$ ssh-copy-id -i ~/.ssh/id_rsa.pub dh_username@server.dreamhost.com
I also like to add the new domain as a Host mapping to the underlying server.dreamhost.com host to the domain name in my SSH config, so I can SSH simply with ssh your-domain.example.com without needing to remember the username. e.g. on your local machine add to ~/.ssh/config:
Host your-domain.example.com
HostName server.dreamhost.com
User dh_username
SSH into your new domain.
$ ssh your-domain.example.com
In your home directory, create a new Python3 virtualenv:
$ python3 -m virtualenv -p `which python3` env
Activate the venv.
$ . env/bin/activate
Install Django, and create your new Django project.
$ pip install django
$ python -m django startproject app
You can use something other than "app" for your project name if you like, but remember to replace "app" with your project name in the commands/config below.
You should now have in your home directory subdirectories of env, app and www, for the virtualenv, Django project Python code, and static files respectively.
Before your Django project will function you need to run the initial migrations to setup the database.
$ cd ~/app
$ python manage.py migrate
Create super user, taking note of the username/password.
$ python manage.py createsuperuser
Now to connect Passenger to you Django project via a WSGI handler.
In your ~/www folder, add a passenger_wsgi.py file with these contents:
#!/home/dh_username/env/bin/python3
import sys, os
# Switch to the virtualenv if we're not already there
VENV_DIR = '/home/dh_username/env'
APP_DIR = '/home/dh_username/app'
INTERP = VENV_DIR + "/bin/python3"
if sys.executable != INTERP:
os.execl(INTERP, INTERP, *sys.argv)
sys.path.append(APP_DIR)
sys.path.append(APP_DIR + '/app')
sys.path.insert(0, VENV_DIR + '/bin')
sys.path.insert(0, VENV_DIR + '/lib/python3.6/site-packages')
os.environ['DJANGO_SETTINGS_MODULE'] = 'app.settings'
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
Note: Replace the things highlighted in BOLD with the appropriate things for your username/app.
That is, replace dh_username with your username , and replace app with your project name, if you didn't use "app".
This script ensures that if the WSGI handler is not invoked with the venv's interpreter, it re-invokes with the venv's interpreter and thus the pip packages installed in the venv.
Make that executable:
$ chmod +x ~/www/passenger_wsgi.py
Open your ~/app/app/settings.py and add your site's domain name to the ALLOWED_HOSTS list.
We also need to configure the STATIC_ROOT. This is the path that Django collectstatic management command puts the static assets in. These are the CSS/JS files needed by the Django admin interface. These need to be in a subdirectory of your Web Directory, so Apache can serve them. So they need to be in a subdirectory of ~/www/public if you used "www/public" as the Web Directory when you setup the domain in the DreamHost control panel.
So add to your ~/app/app/settings.py after the definition of STATIC_URL:
STATIC_ROOT = '/home/dh_username/www/public' + STATIC_URL
Note that you need to update dh_username to match your username, and www to match your Web Directory.
Then build the static assets, and put them there.
$ python manage.py collectstatic
This will copy the static assets required by the Django admin interface into ~/www/public/static/ (assuming your STATIC_URL is the default of "/static/").
Add a simple ~/www/public/index.html.
Your domain should almost be ready to respond to web requests. The final step is to let the Passenger handler know that your WSGI handler code has changed. Passenger monitors the timestamp on the ~/www/tmp/restart.txt file, and every time the timestamp changes it reloads the handler - your Django Python project.
To to make the web server notice your new handler code:
$ mkdir ~/www/tmp/
$ touch ~/www/tmp/restart.txt
Now open your domain's index.html in your web browser. Hopefully if everything's working, you should see your placeholder index.html.
Now open your https://$YOUR_DOMAIN/admin in your web browser, and you should see the Django admin UI, complete with styles.
If you don't see the styles, it's likely your STATIC_ROOT or STATIC_URL isn't correct.
If you do see the Django admin, then you know that Django is working, and you can now start adding other views. Victory!
Note, before you publicise your size, you should turn off DEBUG=True in your project's settings.py!
Also be aware that every time you change your Python code, you need to touch ~/www/tmp/restart.txt, to tell Passenger to reload your Python code. Without this step, Passenger may not restart the already running WSGI handler, and so you'll end up running the old in-memory code, rather than the new code you just updated sitting on disk.
So as part of your deploy process, you need to ensure you touch ~/www/tmp/restart.txt.
Normal procedure these days is to put secrets in environment variables, but I couldn't figure out a satisfactory way to inject environment variables into the Passenger/WSGI process with DreamHost. So I stored the secrets (like the CSRF secret key, email login details) and settings that vary between production and development in a settings.json file which I keep in the ~/app/ dir. My settings.py loads this file, and there are fields in that file that control things that vary between my local dev environment and production. For example I have fields which control the value of DEBUG and ALLOWED_HOSTS, and on my dev machine the settings.json sets DEBUG=True and adds "localhost" to the ALLOWED_HOSTS. Whereas on production, these values should default to safe values. I don't commit my settings.json into my git repository, so that way I am not committing secrets.
So with that, you should now have a working Django project on DreamHost running Python 3 in a virtualenv!
Firstly, a quick explainer on the end-goal of how web requests will be routed through the server.
DreamHost uses Apache to handle HTTP requests. We want to use Python to handle some requests, but use Apache to serve the static files (HTML, CSS, etc), as using Python for a general purpose server would not be performant enough. DreamHost recommends and prefers the use of Passenger to handle web requests for Python. Passenger was designed for Ruby, but can be used for Python as well.
Your user's web browser makes requests to your DreamHost's server. The server routes the requests to Passenger, which in turn routes them to your WSGI handler - which will be a Django Project in our case. If the WSGI handler doesn't handle the request, it gets routed back to the Apache to handle. For example requests for a static asset (i.e. an HTML or CSS file) would go to Passenger, not be handled, and then fall back to be handled by the web server. When Passenger is invoked, it will start up an instance of your Python code in a stand alone process for handling requests. This process will be kept alive for some time, and any further requests that come in for the next wee while will be serviced by this process. So your process may be re-used for multiple requests, but you can't guarantee that it will, as Passenger will shutdown your process after some time period of inactivity.
Passenger will by default start up the Python WSGI handler using the system default Python, which at the time of writing on my DreamHost server was still Python 2. In order to get Python 3 inside our virtualenv, we'll need to re-invoke with our virtualenv's Python 3 interpreter.
You don't want your code to be web accessible. So in your home directory you'll create a directory for your code, a directory for your static assets (HTML/CSS files) that Apache uses as the root, and a directory for your Python 3 virtualenv and pip dependencies.
For a database, I simply used SqlLite, with the database file stored inside the ~/app directory. That was easy to setup and is fine for a simple learning project. I never figured out how to get Python 3 to talk to DreamHost's MySQL server; IIRC the native libraries required for Python 3 to talk to MySQL weren't installed on the server I'm on. I expect if you emailed DreamHost's support team they could install the packages for you to enable Python 3 to talk to their MySQL server. That said, for small workloads, I've found SqlLite to perform better than a full-blown MySQL server.
With that all said, let's dive into the specifics of setting this up.
From the DreamHost control panel, create a new domain website under Domains > Manage Domains. Make sure you turn on the "Enable Passenger" option, and follow the advice about the web directory ending in "public". Make a note of the username, password and server that you were assigned. I'm going to assume your Web Directory is www/public in the examples below.
I'll use the convention that anything that could vary based on your setup that you may need to change in the commands/config below will be in bold. For example the username that DreamHost assigns to your new domain, or the Web Directory.
Still in the Control Panel, under FTP & SSH > Manage Users, select your new domain's user, and enable SSH.
Under Domains > SSL/TLS Certificates, select your new domain, and add a free Lets Encrypt certificate. This will mean you'll be able to use HTTPS on your site.
For convenience, I would then follow DreamHost's instructions to enable passwordless login via SSH keys. For me this was as simple as:
$ ssh-copy-id -i ~/.ssh/id_rsa.pub dh_username@server.dreamhost.com
I also like to add the new domain as a Host mapping to the underlying server.dreamhost.com host to the domain name in my SSH config, so I can SSH simply with ssh your-domain.example.com without needing to remember the username. e.g. on your local machine add to ~/.ssh/config:
Host your-domain.example.com
HostName server.dreamhost.com
User dh_username
SSH into your new domain.
$ ssh your-domain.example.com
In your home directory, create a new Python3 virtualenv:
$ python3 -m virtualenv -p `which python3` env
Activate the venv.
$ . env/bin/activate
Install Django, and create your new Django project.
$ pip install django
$ python -m django startproject app
You can use something other than "app" for your project name if you like, but remember to replace "app" with your project name in the commands/config below.
You should now have in your home directory subdirectories of env, app and www, for the virtualenv, Django project Python code, and static files respectively.
Before your Django project will function you need to run the initial migrations to setup the database.
$ cd ~/app
$ python manage.py migrate
Create super user, taking note of the username/password.
$ python manage.py createsuperuser
Now to connect Passenger to you Django project via a WSGI handler.
In your ~/www folder, add a passenger_wsgi.py file with these contents:
#!/home/dh_username/env/bin/python3
import sys, os
# Switch to the virtualenv if we're not already there
VENV_DIR = '/home/dh_username/env'
APP_DIR = '/home/dh_username/app'
INTERP = VENV_DIR + "/bin/python3"
if sys.executable != INTERP:
os.execl(INTERP, INTERP, *sys.argv)
sys.path.append(APP_DIR)
sys.path.append(APP_DIR + '/app')
sys.path.insert(0, VENV_DIR + '/bin')
sys.path.insert(0, VENV_DIR + '/lib/python3.6/site-packages')
os.environ['DJANGO_SETTINGS_MODULE'] = 'app.settings'
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
Note: Replace the things highlighted in BOLD with the appropriate things for your username/app.
That is, replace dh_username with your username , and replace app with your project name, if you didn't use "app".
This script ensures that if the WSGI handler is not invoked with the venv's interpreter, it re-invokes with the venv's interpreter and thus the pip packages installed in the venv.
Make that executable:
$ chmod +x ~/www/passenger_wsgi.py
Open your ~/app/app/settings.py and add your site's domain name to the ALLOWED_HOSTS list.
We also need to configure the STATIC_ROOT. This is the path that Django collectstatic management command puts the static assets in. These are the CSS/JS files needed by the Django admin interface. These need to be in a subdirectory of your Web Directory, so Apache can serve them. So they need to be in a subdirectory of ~/www/public if you used "www/public" as the Web Directory when you setup the domain in the DreamHost control panel.
So add to your ~/app/app/settings.py after the definition of STATIC_URL:
STATIC_ROOT = '/home/dh_username/www/public' + STATIC_URL
Note that you need to update dh_username to match your username, and www to match your Web Directory.
Then build the static assets, and put them there.
$ python manage.py collectstatic
This will copy the static assets required by the Django admin interface into ~/www/public/static/ (assuming your STATIC_URL is the default of "/static/").
Add a simple ~/www/public/index.html.
Your domain should almost be ready to respond to web requests. The final step is to let the Passenger handler know that your WSGI handler code has changed. Passenger monitors the timestamp on the ~/www/tmp/restart.txt file, and every time the timestamp changes it reloads the handler - your Django Python project.
To to make the web server notice your new handler code:
$ mkdir ~/www/tmp/
$ touch ~/www/tmp/restart.txt
Now open your domain's index.html in your web browser. Hopefully if everything's working, you should see your placeholder index.html.
Now open your https://$YOUR_DOMAIN/admin in your web browser, and you should see the Django admin UI, complete with styles.
If you don't see the styles, it's likely your STATIC_ROOT or STATIC_URL isn't correct.
If you do see the Django admin, then you know that Django is working, and you can now start adding other views. Victory!
Note, before you publicise your size, you should turn off DEBUG=True in your project's settings.py!
Also be aware that every time you change your Python code, you need to touch ~/www/tmp/restart.txt, to tell Passenger to reload your Python code. Without this step, Passenger may not restart the already running WSGI handler, and so you'll end up running the old in-memory code, rather than the new code you just updated sitting on disk.
So as part of your deploy process, you need to ensure you touch ~/www/tmp/restart.txt.
Normal procedure these days is to put secrets in environment variables, but I couldn't figure out a satisfactory way to inject environment variables into the Passenger/WSGI process with DreamHost. So I stored the secrets (like the CSRF secret key, email login details) and settings that vary between production and development in a settings.json file which I keep in the ~/app/ dir. My settings.py loads this file, and there are fields in that file that control things that vary between my local dev environment and production. For example I have fields which control the value of DEBUG and ALLOWED_HOSTS, and on my dev machine the settings.json sets DEBUG=True and adds "localhost" to the ALLOWED_HOSTS. Whereas on production, these values should default to safe values. I don't commit my settings.json into my git repository, so that way I am not committing secrets.
So with that, you should now have a working Django project on DreamHost running Python 3 in a virtualenv!
No comments:
Post a Comment