⚠️ This article is outdated and discontinued since Heroku decided to no longer offer their free tiers as this article series suggests to use in August, 2022. Please see this post for details. ⚠️
In the previous part of this series, I tried to give you a brief yet thorough introduction to hosting your projects with Heroku.
That part was special because it was a completely optional part of this series; if you prefer to host your applications on a different platform and skipped that article, I’d like to repeat that this is completely OK and that I had shown nothing you will need for anything different but interacting with Heroku. You will hopefully notice no blank spots in the following articles. There is no need to read that article if you do not plan to use Heroku for hosting your bot. But you should be familiar enough with your hosting solution of choice to adopt the Heroku – commands I show here to an adequate setup for your hosting solution.
Today we will finally start creating our bot with Django. What we did up until now was just some kind of preparation and establishing background. In this part, we will finally start with the real stuff. ?
Foreword
This article reflects how I created my first Django project. I had some experience with Python before already, but I never created any real Django project apart from the standard tutorial – setups, when one is trying to follow the official Django intro, for example.
When you follow this article, you will end up with a working Telegram bot which I have not found any issue with yet. But neither is it based on any best-practice on how to create Django applications nor will there be any conceptual groundbreaking insights. Maybe you could not even do worse than to follow my instructions when you are unfamiliar with Django and want to learn how it is done; maybe, I do not know better myself. But on the other hand I think I have created something not too bad since it is working as expected, but I doubt that what I’m about to show with this article will be a perfect first attempt.
Because of that, I encourage you to try to get the most out of this article, even if it isn’t perfect and nobody with fundamental Django skills would consider it to be a great start into Django, most certainly. But do not take anything in this article blindly as the one and only truth.
If you are more familiar with Django or do have better ideas with any of the shown codes or concepts, please let me know in the comments. I would really appreciate some feedback; positive or negative. I will adapt valuable advice by updating this article, accordingly.
Providing a proper environment
In this project, we will use the following:
- Django
- For our Telegram Bot, we will use Django 2.1.
- Database
- To not make the local setup too complicated, we will use a sqlite3 database for local development and testing.
- For the production application in Heroku, we will use a PostgreSQL database, since this is a great SQL RDBMS and one of the three managed data services, which are available to all customers in Heroku.
If you want to learn more about PostgreSQL in Heroku please read Herokus article on heroku-postgres.
- Python
- Since Heroku’s current default Python is 3.6.6 (as of 2018-08-24), I will also use 3.6; it’s not necessary to exactly match the patch level version.
- If you want to use Python 3.7 for any reason (utilization of Data Classes for example), you can do so, but you will need an additional step, not explicitly mentioned later on anymore:
To define Python 3.7 as the desired Python runtime in your project, add python-3.7.0 to a file called runtime.txt inside your project folder (echo ‘python-3.7.0’ > runtime.txt). Heroku will automatically pick that up and deploy your dynos with this runtime. More on Python runtimes on Heroku in python-runtimes article.
To match this environment, it is recommended to create a new virtualenv for it. I will show how to do this using pipenv in a minute.
Preparing your project
In the previous part of this series (which was optional), we created a new app on Heroku and initialized a simple Flask application in ~/tbot. We do not need any of that in the upcoming parts since it was meant to demonstrate Heroku’s workflow, exclusively. If you followed that article, you may leave everything in place like it is. If it isn’t used (no one is accessing its URL), it won’t consume any resources.
For this article, we will initiate a new project folder at ~/dtbot. If you prefer another location, feel free to do so, but please adopt that in the following occurrences of ~/dtbot accordingly.
First, we need to have this empty folder created, initialize Git in it and create a fresh virtualenv using pipenv :
~ $ mkdir ~/dtbot ~ $ cd ~/dtbot ~/dtbot $ git init Initialized empty Git repository in /home/testuser/dtbot/.git/ ~/dtbot $ echo 'db.sqlite3' > .gitignore ~/dtbot $ echo '**/__pycache__/' >> .gitignore ~/dtbot $ echo '*.pyc' >> .gitignore ~/dtbot $ pipenv --python python3.6.6 Creating a virtualenv for this project… Using /opt/python/python3.6.6/bin/python3.6.6 (3.6.6) to create virtualenv… ⠋Running virtualenv with interpreter /opt/python/python3.6.6/bin/python3.6.6 Using base prefix '/opt/python/python3.6.6' New python executable in /home/testuser/.virtualenvs/dtbot-hT9CNosh/bin/python3.6.6 Also creating executable in /home/testuser/.virtualenvs/dtbot-hT9CNosh/bin/python Installing setuptools, pkg_resources, pip, wheel...done. Virtualenv location: /home/testuser/.virtualenvs/dtbot-hT9CNosh Creating a Pipfile for this project… ~/dtbot $
Next, we will install some required Python modules:
~/dtbot $ pipenv install django telepot gunicorn Installing django… Collecting django Using cached https://files.pythonhosted.org/packages/51/1a/e0ac7886c7123a03814178d7517dc822af0fe51a72e1a6bff26153103322/Django-2.1-py3-none-any.whl ... Updated Pipfile.lock (dcba73)! Installing dependencies from Pipfile.lock (dcba73)… ? ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 13/13 — 00:00:03 To activate this project's virtualenv, run the following: $ pipenv shell ~/dtbot $
For those of you who are using Heroku to host your Django bot, install an additional module called django-heroku:
~/dtbot $ pipenv install django-heroku Installing django-heroku… ...
django-heroku automatically pulls in some additional modules:
- dj-database-url
- psycopg2
- whitenoise
With these modules installed, we can simplify and optimize the Django integration with Heroku; please read the project page for details. I will explain how to utilize these shortly.
⚠ Attention:
I’m using pipenv in this guide to populate my virtualenv with modules. This automatically creates a file called Pipfile.
Heroku’s buildpacks detect and build scripts are checking if a file called Pipfile or requirements.txt exists in the root of your project to detect the proper buildpack to be Python.
If you do not use pipenv to manage your environment, like plain virtualenv + pip or no virtualenv at all, you need to create a requirements.txt file manually whenever you change your module composition in your project like this:
(dtbot-hT9CNosh) ~/dtbot $ pip freeze > requirements.txt
This not only allows Heroku’s build process to identify the proper buildpack as Python, but it also tells that build process with which modules it needs to populate the hosting environment with to make your app work. Remember: Every change in your code results not only in the code to be exchanged and some WSGI server to be restarted, but a completely new environment will be created from zero. Heroku needs to know what modules to include in this build for your container; having the Pipfile or requirements.txt in place is the way you do this.
Create a new Heroku app
While you are inside your local project folder (with the freshly initialized Git repo in it), create a new app in your Heroku environment (for details on this process, please read the previous part of this series):
(dtbot-hT9CNosh) ~/dtbot $ heroku create --buildpack heroku/python Setting buildpack to heroku/python... done Creating app... done, ⬢ dry-tundra-61874 https://dry-tundra-61874.herokuapp.com/ | https://git.heroku.com/dry-tundra-61874.git (dtbot-hT9CNosh) ~/dtbot $ git remote -v heroku https://git.heroku.com/dry-tundra-61874.git (fetch) heroku https://git.heroku.com/dry-tundra-61874.git (push) (dtbot-hT9CNosh) ~/dtbot $
We will need this, soon. The –buildpack heroku/python part is optional, yet recommended, since it explicitly sets our buildpack (more on this later) to be Python. Otherwise Heroku would try to figure the proper one out by itself. This also works very well, but since we are Pythonistas, let’s keep it with ? The Zen of Python ?:
Explicit is better than implicit.
Activate your virtualenv and initiate your project
We can activate our virtualenv now. I recommend to do it like this:
~/dtbot $ source $(pipenv --venv)/bin/activate (dtbot-hT9CNosh) ~/dtbot $
This last command is a bit special in my personal workflow: I do not like the pipenv default way to spawn a sub-shell using pipenv shell. So I’m activating the virtualenv pipenv creates by sourcing it’s “activate”-script the classical virtualenv-way.
Now, the time has come to initiate your Django project:
(dtbot-hT9CNosh) ~/dtbot $ django-admin startproject dtbot . (dtbot-hT9CNosh) ~/dtbot $ ls -l total 28 -rw-rw-r-- 1 testuser testuser 241 Aug 24 15:58 Pipfile -rw-rw-r-- 1 testuser testuser 13515 Aug 24 15:59 Pipfile.lock drwxrwxr-x 2 testuser testuser 4096 Aug 24 16:15 dtbot -rwxrwxr-x 1 testuser testuser 537 Aug 24 16:15 manage.py (dtbot-hT9CNosh) ~/dtbot $ ls dtbot/ __init__.py settings.py urls.py wsgi.py (dtbot-hT9CNosh) ~/dtbot $
Let’s calibrate some settings in ~/dtbot/dtbot/settings.py to match Heroku’s recommended implementation first:
- Add import django_heroku to the top of your settings.py file.
Then, to the very bottom of it, add django_heroku.settings(locals()). By this, some sensitive credentials (like the database credentials) and secrets (like SECRET_KEY) and general settings (like DATABASE_URL) do not need to be stored in the settings.py file, but are configured using environment variables inside of the dyno or work out-of-the-box because Heroku provides these values in environment variables automatically.
This adds some value to your project since you will need to configure less in files. Also, not even your co-workers will get these secrets exposed, your app will become re-usable without the need to make any change to its settings.py file, et cetera. Even if there are no co-workers, maybe there will be some in the future or you decide to make your bot’s code publically available on Github or similar; you can’t know today what will happen in the future. Because of that, adding sensitive information to a VCS like Git is not recommended generally.
To learn about the details of this, please read Herokus django-app-configuration article. - Add import os to the top of settings.py. Then replace DEBUG = True by this:
DEBUG = os.environ.get('DEBUG') if DEBUG is None: DEBUG = False else: if 'True' in DEBUG: DEBUG = True else: DEBUG = FalseI will explain the logic behind this in a minute.
- Add 127.0.0.1 to ALLOWED_HOSTS , so you can access your Django locally.
As a last initialization step, apply any migrations now; these will be applied to the default sqlite3 DB file db.sqlite3 in your project folder. Please execute this following command blindly, so far, even though I will not explain what it does or what a migration is right now, since that is too wide out of scope, right now. For now, you only need to know that this creates a file named sqlite3.db inside your Django project-root directory, which stores a complete database layout, which this command creates in there. I will pick this up and explain it more deeply in Part 8 of this series:
(dtbot-hT9CNosh) ~/dtbot $ python manage.py migrate Operations to perform: Apply all migrations: admin, auth, contenttypes, sessions Running migrations: Applying contenttypes.0001_initial... OK Applying auth.0001_initial... OK Applying admin.0001_initial... OK Applying admin.0002_logentry_remove_auto_add... OK Applying admin.0003_logentry_add_action_flag_choices... OK Applying contenttypes.0002_remove_content_type_name... OK Applying auth.0002_alter_permission_name_max_length... OK Applying auth.0003_alter_user_email_max_length... OK Applying auth.0004_alter_user_username_opts... OK Applying auth.0005_alter_user_last_login_null... OK Applying auth.0006_require_contenttypes_0002... OK Applying auth.0007_alter_validators_add_error_messages... OK Applying auth.0008_alter_user_username_max_length... OK Applying auth.0009_alter_user_last_name_max_length... OK Applying sessions.0001_initial... OK (dtbot-hT9CNosh) ~/dtbot $ ls -l db.sqlite3 -rw-r--r-- 1 testuser testuser 40960 Aug 24 16:59 db.sqlite3 (dtbot-hT9CNosh) ~/dtbot $
That’s it so far – let’s test this and play around with it a bit next.
Start the build-in HTTP server and enable DEBUG
Let’s test drive our setup so far. Run your app now like this:
(dtbot-hT9CNosh) ~/dtbot $ python manage.py runserver Performing system checks... System check identified no issues (0 silenced). August 24, 2018 - 15:02:58 Django version 2.1, using settings 'dtbot.settings' Starting development server at http://127.0.0.1:8000/ Quit the server with CONTROL-C.
When you point your browser to http://127.0.0.1:8000/, you should notice two things:
- This should work and a web page should be shown.
- The content of the web page should be:
If you ever had worked with Django before, you may have expected a DEBUG – page with additional details. This is not shown since we changed DEBUG = True in the settings.py file for our environment extraction logic before. By default (when there is no DEBUG variable with a value of True assigned to it) this results in False as a security measure.
To have the DEBUG – page shown here instead set the environment variable DEBUG to True before starting the HTTP server using python manage.py runserver like this:(dtbot-hT9CNosh) ~/dtbot $ export DEBUG="True" (dtbot-hT9CNosh) ~/dtbot $ python manage.py runserver Performing system checks... System check identified no issues (0 silenced). August 24, 2018 - 15:12:09 Django version 2.1, using settings 'dtbot.settings' Starting development server at http://127.0.0.1:8000/ Quit the server with CONTROL-C.
Now, when you refresh your browser, the DEBUG page should be shown instead:
Heroku config vars
So – why did we change the way DEBUG is switched now?
In containerized environments, settings are often defined using environment variables inside the containers, since this enables the users to use reusable applications without changing any files in each container. It’s just more flexible and easier to orchestrate than config file settings, since scaling and multi-node setups are much more a standard pattern in containerization than it is in the bare metal world. Heroku is no exception to this.
You can change and list environment variables for an application in Heroku using the heroku CLI. To show all registered config vars and their value, just execute heroku config. New ones can be set or existing ones can be changed using heroku config:set:
(dtbot-hT9CNosh) ~/dtbot $ heroku config === dry-tundra-61874 Config Vars (dtbot-hT9CNosh) ~/dtbot $ heroku config:set DEBUG=True Setting DEBUG and restarting ⬢ dry-tundra-61874... done, v3 DEBUG: True (dtbot-hT9CNosh) ~/dtbot $ heroku config === dry-tundra-61874 Config Vars DEBUG: True (dtbot-hT9CNosh) ~/dtbot $
Now, whenever you push a change to your project, it will be deployed with Debugging enabled.
? Most certainly, you do not want that for a productive application ?! There are more advanced methods to debug a remote Django web application than to expose it’s debugging data to the world!
But for this tutorial-like application, it is a great and easy example to demonstrate how this concept is working on Heroku.
Leave this set to True; we will push our Django app to Heroku soon. Then, we will play around with this a bit and finally switch the content of the DEBUG config var to False, before we will never touch it again ?
Prepare our Django app to be shipped ?
Before we can deploy our app on Heroku, we need to do two more preparations:
Add a PostgreSQL RDBMS to your Heroku app ?
In Heroku, this is only one command:
(dtbot-hT9CNosh) ~/dtbot $ heroku addons:add heroku-postgresql Creating heroku-postgresql on ⬢ dry-tundra-61874... free Database has been created and is available ! This database is empty. If upgrading, you can transfer ! data from another database with pg:copy Created postgresql-rectangular-59399 as DATABASE_URL Use heroku addons:docs heroku-postgresql to view documentation (dtbot-hT9CNosh) ~/dtbot $
That’s it!
When you check your app’s config vars, you will notice that a new variable called DATABASE_URL was added to your application. Also, it now shows up in the list of addons:
(dtbot-hT9CNosh) ~/dtbot $ heroku addons
Add-on Plan Price State
──────────────────────────────────────────────── ───────── ───── ───────
heroku-postgresql (postgresql-rectangular-59399) hobby-dev free created
└─ as DATABASE
The table above shows add-ons and the attachments to the current app (dry-tundra-61874) or other apps.
(dtbot-hT9CNosh) ~/dtbot $ heroku config
=== dry-tundra-61874 Config Vars
DATABASE_URL: postgres://${PG_USER}:${PG_PASS}@ec2-54-83-51-78.compute-1.amazonaws.com:5432/${PG_DBNAME}
DEBUG: True
(dtbot-hT9CNosh) ~/dtbot $
This way, your Django app is connected to your PostgreSQL database automatically, since the connection string defined in DATABASE_URL is picked up by the django_heroku module we added to our settings.py file earlier.
This way, you are done preparing a PostgreSQL database for your Heroku hosted web app with just one single command ?
Isn’t this just awesome?
But there is even more: You can even connect to that DB by extracting the necessary parts for your application (pgAdminIII, psql, …) from that URL. If you want to use psql from your current workstation, there is even a more convenient way:
(dtbot-hT9CNosh) ~/dtbot $ heroku pg:psql --> Connecting to postgresql-rectangular-59399 psql (10.5 (Ubuntu 10.5-1.pgdg16.04+1), server 10.4 (Ubuntu 10.4-2.pgdg14.04+1)) SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, bits: 256, compression: off) Type "help" for help. dry-tundra-61874::DATABASE=>
So, in theory, whenever you need a PostgreSQL DB for a project hosted literally anywhere, you could just create an empty Heroku app, add an heroku-postgres addon to it and start utilizing that DB.
To me, this is pure elegance!
Create a Procfile
Finally, as the last preparation step, let’s get one more detail out of the way: Create a Procfile to configure how Heroku should startup your dyno with but one line as content:
web: gunicorn dtbot.wsgi
If you want to know more about a Procfile, please read the previous part of this series or the corresponding Heroku article on the Procfile.
Ship your app! ?
We now should have the basics set and be ready to commit our files to the Git repository and to push it to the heroku remote to trigger a deployment.
? Once more a warning ?: Remember that since we still have the config var DEBUG set to True, our app will be in debugging mode! We do not really have valuable data in it yet and it is also more than unlikely that in the few minutes between our deployment and the moment when we switch it to False someone with bad intention will visit our unpublished, randomly generated URL. But please decide this on your own. If you do not feel comfortable with this and do not need to see the debug page on your own environment to get an idea for the environment, better switch config var DEBUG to False before you do your push to heroku master.
Deploy your prepared Django app like this:
(dtbot-hT9CNosh) ~/dtbot $ git add . (dtbot-hT9CNosh) ~/dtbot $ git commit -m "Init" [master (root-commit) 3b3927c] Init 13 files changed, 440 insertions(+) create mode 100644 .gitignore create mode 100644 Pipfile create mode 100644 Pipfile.lock create mode 100644 Procfile create mode 100644 dtbot/__init__.py create mode 100644 dtbot/settings.py create mode 100644 dtbot/urls.py create mode 100644 dtbot/wsgi.py create mode 100755 manage.py (dtbot-hT9CNosh) ~/dtbot $ git push heroku master Counting objects: 24, done. Delta compression using up to 4 threads. Compressing objects: 100% (19/19), done. Writing objects: 100% (24/24), 10.99 KiB | 0 bytes/s, done. Total 24 (delta 3), reused 0 (delta 0) remote: Compressing source files... done. remote: Building source: remote: remote: -----> Python app detected remote: -----> Installing python-3.6.6 remote: -----> Installing pip remote: -----> Installing dependencies with Pipenv 2018.5.18… remote: Installing dependencies from Pipfile.lock (ce9952)… remote: -----> Installing SQLite3 remote: -----> $ python manage.py collectstatic --noinput remote: /app/.heroku/python/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>. remote: """) remote: 119 static files copied to '/tmp/build_eb69259fca4b3ef8687a35ccdf98d226/staticfiles', 375 post-processed. remote: remote: -----> Discovering process types remote: Procfile declares types -> web remote: remote: -----> Compressing... remote: Done: 64.8M remote: -----> Launching... remote: Released v5 remote: https://dry-tundra-61874.herokuapp.com/ deployed to Heroku remote: remote: Verifying deploy... done. To https://git.heroku.com/dry-tundra-61874.git * [new branch] master -> master (dtbot-hT9CNosh) ~/dtbot $
“remote: … deployed to Heroku” – exactly what we were hoping for to see! When we navigate to our apps URL (https://dry-tundra-61874.herokuapp.com/ in this example), we should now see the DEBUG page we did before when we tested locally. For convenience, you can simply execute heroku open to direct your browser to the correct URL:
… let’s disable the debugging now to make that bad tummy feeling go away before we proceed.
In your shell, simply execute heroku config:set DEBUG=False :
(dtbot-hT9CNosh) ~/dtbot $ heroku config:set DEBUG=False Setting DEBUG and restarting ⬢ dry-tundra-61874... done, v6 DEBUG: False (dtbot-hT9CNosh) ~/dtbot $
With immediate effect, the debugging mode should be disabled when you refresh your browser:
Phew, that happened to become quite a long article, hasn’t it?
Because of this, I will stop now and better continue in the next part.
So far, you managed to create your Django project skeleton, prepare it for Heroku and deployed it already! That is enough for one day anyway, I guess.
Outlook for the next part of the series
We just learned how to prepare a proper local environment for Django and Heroku by creating a virtualenv using pipenv. Also, we saw how to kickstart a new Django project, make changes to its config appropriate for Heroku hosting, deploy it to Heroku and how to use Herokus config vars properly to quickly change settings in your running and future deployed apps. Finally, we already had a peek into some of Herokus convenient development features and how to utilize them by the heroku CLI.
In the next article of this series, we will see some more of Herokus features which support you in your development. We also need to do some other actions to finalize our Django app in Heroku; we neither have applied our migrations to the PostgreSQL DB, nor created a superuser to access the Admin Area yet. Finally, we will create a Django app which can receive the JSON data sent by your Telegram bot and do something with it.
If you liked or disliked this article, I’d love to read that in the comments!
Enjoy coding!
Born in 1982, Marc Richter is an IT enthusiast since 1994. He became addicted when he first put his hands on their family’s PC and never stopped investigating and exploring new things since then.
He is married to Jennifer and a proud father of two wonderful children.
His current professional focus is DevOps and Python development.
An exhaustive bio can be found in this blog post.