Running Django Management Commands Periodically by Using CRONTAB Linux Command

• I will not explain what the crontab is in this article. In a nutshell it is a tool that helps us to create periodic tasks on linux.

• If you run crontab --help it prints out the parameters we can use

• adnan @ adnankaya ~
   └─ λ crontab --help
  crontab: invalid option -- '-'
  crontab: usage error: unrecognized option
  usage:  crontab [-u user] file
          crontab [ -u user ] [ -i ] { -e | -l | -r }
                  (default operation is replace, per 1003.2)
          -e      (edit user's crontab)
          -l      (list user's crontab)
          -r      (delete user's crontab)
          -i      (prompt before deleting user's crontab)
  

• The following command will open nano editor and inside of it you will see some comments for guidence.

• xxxxxxxxxx
  adnan @ adnankaya ~
   └─ λ EDITOR=nano crontab -e
  

• The nano crontab comments and description

• ​x
  GNU nano 4.8                                                                                /tmp/crontab.abcDef/crontab                                                                                          
  # Edit this file to introduce tasks to be run by cron.
  # 
  # Each task to run has to be defined through a single line
  # indicating with different fields when the task will be run
  # and what command to run for the task
  # 
  # To define the time you can provide concrete values for
  # minute (m), hour (h), day of month (dom), month (mon),
  # and day of week (dow) or use '*' in these fields (for 'any').
  # 
  # Notice that tasks will be started based on the cron's system
  # daemon's notion of time and timezones.
  # 
  # Output of the crontab jobs (including errors) is sent through
  # email to the user the crontab file belongs to (unless redirected).
  # 
  # For example, you can run a backup of all your user accounts
  # at 5 a.m every week with:
  # 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
  # 
  # For more information see the manual pages of crontab(5) and cron(8)
  # 
  # m h  dom mon dow   command
  
  

• Let's edit for prettier comment

  xxxxxxxxxx
  # (Sunday to Saturday; 7 is also Sunday on some systems)
  # ┌────────────> minute (0 - 59)
  # │ ┌─────────────> hour (0 - 23)
  # │ │ ┌─────────────> day of month (1 - 31)
  # │ │ │ ┌─────────────> month (1 - 12)
  # │ │ │ │ ┌─────────────> day of week (0 - 6) (Sunday to Saturday)
  # │ │ │ │ │                                       
  # │ │ │ │ │
  # │ │ │ │ │
  # v v v v v
  # m h d M W   command
  # * * * * *  command_to_execute
  
  # The following crontab runs every minute
  * * * * * echo "Hello world"
  

• To see the logs of CRON open a new terminal and run $ tail -f /var/log/syslog | grep CRON

• Output

  xxxxxxxxxx
  adnan @ adnankaya ~/
   └─ λ tail -f /var/log/syslog | grep CRON
  May 25 02:52:01 adnankaya CRON[767650]: (adnan) CMD (echo "Hello world") # Notice 02:52
  May 25 02:53:01 adnankaya CRON[767720]: (adnan) CMD (echo "Hello world") # Notice 02:53
  May 25 02:54:01 adnankaya CRON[767807]: (adnan) CMD (echo "Hello world") # Notice 02:54
  

 

Django Part

• I do have a command which named as fetch_data . As you guessed it fetches data from external api and stores into database.

• The command file structure like this

• xxxxxxxxxx
  adnan @ adnankaya ~/webdev/backend
   └─ λ tree api/management/
  api/management/
  ├── commands
  │   ├── fetch_data.py
  │   └── __init__.py
  └── __init__.py
  
  1 directory, 3 files
  

• virtualenv -p python3 venv created a new virtualenv instance

• python manage.py makemigrations && python manage.py migrate

 

Crate New Crontab

• xxxxxxxxxx
  adnan @ adnankaya ~
   └─ λ EDITOR=nano crontab -e
  

• Specify the period. I used cd command because I needed to read a json file. Use full path of your python inside virtualenv and use your manage.py full path. Here is my crontab:

• x
  # m h  dom mon dow   command
  #
  # ┌───────────── minute (0 - 59)
  # │ ┌───────────── hour (0 - 23)
  # │ │ ┌───────────── day of month (1 - 31)
  # │ │ │ ┌───────────── month (1 - 12)
  # │ │ │ │ ┌───────────── day of week (0 - 6) (Sunday to Saturday)
  # │ │ │ │ │
  # │ │ │ │ │
  # * * * * *  command_to_execute
  
  # The following crontab runs every minute
  * * * * * echo "Hello world"
  # The following crontab runs every 30 minute
  */30 * * * * cd /home/adnan/webdev/upwork/CurrencyFetch/backend && /home/adnan/webdev/upwork/CurrencyFetch/backend/venv/bin/python /home/adnan/webdev/upwork/CurrencyFetch/backend/manage.py fetch_data > /tmp/cronlog.txt 2>&1
  #                             ^^^^^ redirect error messages (stderr) to the visible command line (stdout)
  # We can see the error messages insite /tmp/cronglog.txt
  

• To see cronlog.txt

• xxxxxxxxxx
  adnan @ adnankaya ~
   └─ λ cat /tmp/cronlog.txt 
  Process started...
  Process finished...
  

• That's it! Our crontab is running on the background and fetches data every 30 minutes.

Use Cases

• You can fetch some data from external api.
• You can index elasticsearch clusters
• You can backup your database
• and so on..

Resources

• https://crontab.guru/
• Corey Schafer | Linux/Mac Tutorial: Cron Jobs - How to Schedule Commands with crontab
• https://stackoverflow.com/questions/3287038/cron-and-virtualenv