Python Notes: Picture-of-the-Day

  1. The information presented here is intended for educational use.
  2. The information presented here is provided free of charge, as-is, with no warranty of any kind.
  3. Edit: 2021-10-31 (renovated my JavaScript file to better handle async image loading)

Overview

  • my original picture-of-the-day code (see the top right corner of this home page: http://neilrieck.net/) was written entirely in JavaScript. When you passed it a month-day pair in "mmdd" format, it would provide the URL (either local or remote) of one image along with some displayable text and an optional hyperlink to some article.
  • Adding support for more images was a pain so I moved all the logic into a Python program while the data was moved into a MySQL/MariaDB database. The whole thing is designed to return "escaped data" in XML format. Click the next hyperlink to see a simple example for "0212" (Feb-12) which celebrates the birthday of Charles Darwin
    xml http://neilrieck.net/cgi-bin/daily_pix?xml=1&mmdd=0212
    json http://neilrieck.net/cgi-bin/daily_pix?json=1&mmdd=0212
    plain text http://neilrieck.net/cgi-bin/daily_pix?mmdd=0212
  • this new Python application is composed of four files
    1. one html file (a web page to test my code; can cycle from 0101 to 1231)
    2. one java script file (only run client-side)
    3. one bash script (launches my server-side program)
    4. one python file (only run server-side)
    5. database schema
      database: hack1 table: pixofday
      field name use
      mmdd0 first month-day to show a picture
      mmdd1 last month-day to show a picture
      comment text used in debugging
      src local or remote location of the desired image; insert this into an IMG tag
      calc if not blank, then do an anniversary calculation;
      eg. 2019 was the fiftieth anniversary of the Apollo-11 landing in 1969
      text1 preformatted html
      text2 preformatted html (optional)
      text3 preformatted html (optional)
    6. database other:
      1. be sure to enter "0000" and "9999" to ensure that a default is always returned
      2. one of my special entries lays between "8000" and "8099"
  • clicking a button on the web page calls a JavaScript routine
    • one JavaScript routine, named doReset(), works entirely client-side without annoying the server
    • two JavaScript routines, named setDefaults1() and setDefaults2(), will call the server (via AJAX) expecting some sort of form initialization
    • one JavaScript routine, named runDemo(), reads all the fields defined in the HTML form, tests them, then passes their contents on to the server (via AJAX) for processing
  • because this is a simple how-to demo, many of the routines are simplistic (e.g. no reliance on JavaScript libraries: jQuery, AngularJS, ReactJS although I was tempted to do so since my code would have been much smaller)
  • there are numerous ways to pass data back to HTML but here I used XML because all modern browsers contain routines to perform XML parsing
  • I also added JSON support since it has been available with JavaScript-1.5 (ECMAscript-5)

file-1: HTML web page

<!DOCTYPE html>
<html lang="en">
<head>
 <meta content="text/html; charset=utf-8" http-equiv="Content-Type" />
 <meta content="Neil S. Rieck (Waterloo, Ontario, Canada)" name="description" />
 <meta name="viewport" content="width=device-width, initial-scale=1">
 <meta content="#ffffff" name="theme-color">
 <title>Test Pix</title>
 <link href="css/nsr-20170909.css" rel="stylesheet" type="text/css">
 <script src="js/picture-viewer72.js"></script>
</head>
<body lang="en-ca" >
 <h4>Test-Pix</h4>
 <table class="tbl-blue" style="width:100%">
  <tr><td>msg</td><td id="msg"></td></tr>
  <tr><td>src</td><td id="src"></td></tr>
  <tr><td style="width:50px">com</td><td id="comment" style="min-width:90%"></td></tr>
  <tr><td>txt</td><td id="text" style="text-align: center; font-weight: bold;height:280px"></td></tr>
  <tr><td>calc</td><td id="calc"></td></tr>
  <tr><td>mmdd</td><td id="mmdd"></td></tr>
 </table>
<script>
 testYear();
</script>
</body>
</html>

file-2: JavaScript

// =================================================================
// title : picture-viewer72.js
// author: Neil Rieck (Waterloo, Ontario, Canada)
// edit  : 2020-10-29
// edit  : 2021-10-30 (coincidence?)
// notes :
// v4  (2019-11-28) works with my new server-side Python code
// v5  (2020-04-04) code cleanup
// v6  (2020-10-29) moved everything into a namespace
// v71 (2021-10-30) better overlapping AJAX (pulling from slow sites)
// v72 (2021-10-31) XmlDoc is no longer global (what was I thinking?)
// ==================================================================
//
//	start of name-space
//
var nsNSR6 = (function() {
//
'use strict';		// no kid-stuff here
//
//	some private variables
//
var debug=0;		//
var slot=0;		//
var mySrc=[];		// image source
var myTxt=[];		// image text (from payload)
var imageArray=[];	// image data
//
//	store data as well as preload the image
//
function preloadImage(url,txt,slot){
    if (typeof slot == "undefined"){
	var slot=9;
    }
    console.log("preload url: "+url+" slot: "+slot);
    try{
	mySrc[slot] = url;
	myTxt[slot] = txt;
	imageArray[slot] = new Image();
	imageArray[slot].src = url;
    }catch(e){ 
	console.error("preload url: "+url+" slot: "+slot+" error: ",e);
    }
}
//
//	start_ajax (start an AJAX transaction)
//
function start_ajax(msg,slot){
    console.log("start ajax slot: "+slot);
// still some Microsoft browsers out there if (typeof XMLHttpRequest == "undefined") XMLHttpRequest=function(){ try { return new ActiveXObject("Msxml2.XMLHTTP.6.0") } catch (e) {} try { return new ActiveXObject("Msxml2.XMLHTTP.3.0") } catch (e) {} try { return new ActiveXObject("Msxml2.XMLHTTP") } catch (e) {} try { return new ActiveXObject("Microsoft.XMLHTTP") } catch (e) {} throw new Error("This browser does not support XMLHttpRequest or XMLHTTP.") } if (msg != ""){ var xhr=new XMLHttpRequest(); if (xhr != null){ xhr.open("GET", msg, true); // async=true xhr.timeout=3000; // 3 seconds max xhr.ontimeout = function(){ console.log("Timeout:",slot); } xhr.onreadystatechange=function(){ if (xhr.readyState == 4 && xhr.status == 200){ var resp$=xhr.responseText; ajax_xml_extract(resp$); } } console.log("sending ajax request for slot: "+slot) xhr.send(null); // not null for POST } } } // // ajax_xml_extract // function ajax_xml_extract(resp$){ // console.log("ajax-xml-extract raw:",resp$); try{ if (window.DOMParser) { var parser=new DOMParser(); var xml=parser.parseFromString(resp$,"text/xml"); }else{ var xml=new ActiveXObject("Microsoft.XMLDOM"); xml.async=false; xml.loadXML(resp$); } var src = get_xml_data("src",xml); var txt = get_xml_data("text",xml); var slot= get_xml_data("slot",xml); console.log("calling preloadImage for slot: "+slot) preloadImage(src,txt,slot); }catch(e){ console.error("ajax_xml_extract:",e) } } // // does string 'x' represent a positive integer? // function isStrInt(x){ x=x.replace(/\s/g,''); if (x=="") return false; try{ y = parseInt(x); }catch(e){ y = -1; } if(y>0){ return true; }else{ return false; } } function htmlEncode(str) { return String(str).replace(/&/g, '&').replace(//g, '>').replace(/"/g, '"'); } function htmlDecode(str) { return String(str).replace(/</g, '<').replace(/>>/g, '>').replace(/"/g, '"').replace(/&/g, '&'); } // // get_xml_data (look for one item) // function get_xml_data(tag,xml){ var x; try{ x = xml.getElementsByTagName(tag)[0].childNodes[0].nodeValue; }catch(e){ console.error("get-xml-data: ",e); x = null; } return(x); } // // get number of days per month // function monthSize(x){ switch(x) { case 2: return(28); case 9: case 4: case 6: case 11: return(30); default: return(31); } } function pad(num, size) { var s = num+""; while (s.length < size) s = "0" + s; return s; } // // test picture-of-the-day logic for the whole year // caveat: only used in testing // function testYear(){ debug=1; // change global var mm=0; // month var dd=0; // day var mmdd=""; var intervalId = setInterval(function(){ if (mm==0){ mm=1; dd=1; }else{ dd++; limit=monthSize(mm); if (dd>limit){ mm++; dd=1; } } if (mm>12){ clearInterval(intervalId); }else{ mmdd=junk=pad(mm,2)+pad(dd,2); console.log("mmdd:"+mmdd);
document.getElementById("mmdd").innerHTML="mmdd: "+mmdd; msg = "http://neilrieck.net/cgi-bin/daily_pix?xml=1&slot=0&mmdd="+mmdd; start_ajax(msg,0); } }, 1000); } // // get one picture for today // function fetchToday(slot){ var d = new Date(); var mm = d.getMonth(); // 0..11 var dd = d.getDate(); // 1..31 var mmdd= pad(mm+1,2)+pad(dd,2); console.log("fetchToday-mmdd: "+mmdd); var msg = "http://neilrieck.net/cgi-bin/daily_pix?xml=1&slot="+slot+"&mmdd="+mmdd; start_ajax(msg,slot); } // // get default picture // 8000: co2_400kyr // 8001: co2_levels (static-local) // 8002: co2_levels (dynamic-remote) // function fetchDefault(slot,mmdd){ if (typeof mmdd == "undefined"){ var slot=8000; } if ((mmdd<8000)||(mmdd>8002)){ slot=8000; } console.log("fetchDefault-mmdd: "+mmdd+" slot: "+slot); var msg = "http://neilrieck.net/cgi-bin/daily_pix?xml=1&slot="+slot+"&mmdd="+mmdd; start_ajax(msg,slot); } // // end-of-name-space logic // now publish the names of anything referenced externally // return{ preloadImage:preloadImage, fetchToday:fetchToday, fetchDefault:fetchDefault, mySrc:mySrc, myTxt:myTxt, imageArray:imageArray } // end of name-space }()); // end of file

file-3: Python3 (optional launcher)

note: this file is usually located in the cgi-bin directory of your web server
 
#!/usr/bin/python3
# title  : daily_pix
# author : Neil Rieck
# edit   : 2019-11-23
# notes  : this is just a loader
# ------------------------------
import daily_pix_131.py
daily_pix_131.main()

file-4: Python3 (program)

note: in this example, this file is found under cgi-bin (but you could move it anywhere else provided Apache can access it)
#!/usr/bin/python3
'''
===================================================================================
title  : daily_pix_131.py
author : Neil Rieck
history:
2019-11-23 daily_pix_main_100.py - calls daily_pix_mysql.py
2019-12-15 daily_pix_main_101.py - cleanup
2020-04-18 daily_pix_main_102.py - added JSON support
2020-05-09 daily_pix_130.py      - moved db here; cursor now returns a dictionary
2020-07-11                       - tiny fix in anniversary calc
2020-07-11 daily_pix_131.py      - replaced format strings with f strings
purpose:
1) for a given month-day (format: mmdd) return a picture along with associated text
2) optionally, return the data in XML or JSON format
3) these PEP8 rules are disabled: E305,E501 so I can code in a traditional way
===================================================================================
'''
import cgi
import datetime
import json
import sys
#   sudo python3 -m pip install mysql-connector
import mysql.connector as db
#
hostUrl = "http://neilrieck.net"                                    # default URL of this site (read on)
#
#   common function to BAIL (will be visible in the browser's console)
#
def errorExit(msg):
    print("content-type: text/plain; charset=utf-8")
    print("")
    print(msg)
    sys.exit()
#
#   logIt (a simple file-based logger)
#   caveat: if SELinux is enabled then write to /var/log/daily_pix_msg_log.txt
#
def logIt(msg):
    try:
        fn = "daily_pix_msg_log.txt"
        f = open(fn, "a")
        f.write(f"msg: {msg}\r\n")
        f.close()
    except Exception:
        pass
#
#   do that database thing
#
def get_record(parm):
    try:
        con = db.connect(host='127.0.0.1', user='read123', passwd='read123', database='hack1')
        cur = con.cursor(dictionary=True)                           # force return of dict
        cmd = "select * from pixofday "
        cmd += f"where (mmdd1 <= {parm}) and (mmdd2 >= {parm}) "
        cmd += "order by concat(mmdd2,mmdd1) limit 1"
        cur.execute(cmd)
        # row = cur.fetchall()                                      # returns a list of tuples (or dicts)
        row = cur.fetchone()                                        # returns a tuple (or dict)
        return row
    except Exception as e:
        msg = f"Exception: {e} while opening database"
        logIt(msg)
        errorExit(msg)

'''
===============================================================

 <<< general description >>>

 1) database field definitions
   0: mmdd1 (start date)
   1: mmdd2 (end date)
   2: comment
   3: calc  (if not blank then do an anniversary calculation)
   4: src   (image source)
   5: text1 (usually a title associated with the photo)
   6: text2
   7: text3
 2) data:
   1: each record will indicate that a certain photo be shown between mmmdd1 and mmdd2
   2: one special record has mmdd1="0000" and mmdd2="9999" so something will always be returned
 3) logic:
   1: the 'src' field contains an image location like these examples:
      /images/Edmond_Halley.jpg
      https://www.esrl.noaa.gov/gmd/webdata/ccgg/trends/co2_data_mlo.png
   2: since a couple of my websites are using this logic, any entry not containing "http" will need to be
      prefixed with the URL of this site
   3: text1 to text3 contain HTML so merge them into one string then test for the following patterns:
   4: if we detect the string "+xxx+" then replace it with the resultant 'src' string seen at the end of step 2
   5: if we detect the string "+host+" then replace it with the URL of this site
   6: if we detect the string "+calc+" then do an anniversary calculation (eg. 50th anniversary of Apollo-11)
      then replace the target string with the result
   7: return data to caller via XML or JSON (this means the data must be properly escaped)
 4) Cross-Origin Resource Sharing (CORS)
    For AJAX use you must add a few Cross-Origin Resource Sharing (CORS) directives to Apache config like so:
    <Directory "/var/www/html/cgi-bin">
      AllowOverride None
      Options None
      Require all granted
      # 1) CORS for initial AJAX testing.
      Header set Access-Control-Allow-Origin "*"
      # 2) CORS for single second site support (this Apache instance runs on "http://neilrieck.net"
      # but we want to allow AJAX access from "http://www3.sympatico.ca/n.rieck/")
      #Header set Access-Control-Allow-Origin http://www3.sympatico.ca
      #Header set Access-Control-Allow-Headers "Content-Type,Accept"
      #Header merge Vary Origin
      #Header set MyHeader "Neil's Hack"
     </Directory>
===============================================================
'''
def process_data(row):
    imgSrc = row['src']                                             # logic-1
    if (imgSrc.find('http') == -1):                                 # logic-2
        imgSrc = hostUrl + imgSrc                                   #
    allText = row['text1'] + row['text2'] +row['text3']             # logic-3
    temp = allText.find('+xxx+')                                    # logic-4
    if (temp != -1):                                                # if found...
        allText = allText[:temp] + imgSrc + allText[temp+5:]        #
    temp = allText.find('+host+')                                   # logic-5
    if (temp != -1):                                                # if found...
        allText = allText[:temp] + hostUrl + allText[temp+6:]       #
    temp = allText.find('+calc+')                                   # logic-6
    if (temp != -1):                                                # if found...
        year = datetime.datetime.now().strftime('%Y')               #
        try:                                                        #
            anniv = int(row['calc'])                                # eg. ccyy
        except Exception as e:                                      #
            anniv = 1900                                            #
            msg = f"error: {e}"                                     #
            logIt(msg)                                              #
        diff = int(year) - anniv                                    #
        allText = allText[:temp] + str(diff) + allText[temp+6:]     #
    return allText, imgSrc                                          #

def output_xml(row, mmdd, slot):
    allText, imgSrc = process_data(row)                             #
    escText = cgi.escape(allText, allText)                          # logic-7
    print('Content-Type: text/xml; charset=utf-8')                  # we're returning an XML response
    print('')                                                       # end of HTTP header
    print('<?xml version="1.0" encoding="utf-8"?>')                 # start of XML content
    print('<result>')                                               # user-defined by me (see JavaScript)
    print('<status>1</status>')                                     #
    print('<debug>' + mmdd + '</debug>')                            # the parameter passed in
    print('<comment>' + row['comment'] + '</comment>')              #
    print('<calc>' + row['calc'] + '</calc>')                       #
    print('<src>' + imgSrc + '</src>')                              #
    print('<text>' + escText + '</text>')                           #
    print('<slot>' + slot + '</slot>')                              #
    print('</result>')                                              #
#
def output_json(row, mmdd, slot):
    allText, imgSrc = process_data(row)
    print('Content-Type: text/json; charset=utf-8')                 # we're returning an XML response
    print('')                                                       # end of HTTP header
    del row['text3']                                                # enabled new logic
    del row['text2']
    del row['text1']
    row['imgSrc'] = imgSrc                                          # append
    row['text'] = allText                                           # append
    print(json.dumps(row, indent=1))
#
def output_plain(row, mmdd, slot):                                  #
    allText, imgSrc = process_data(row)                             #
    print('Content-Type: text/plain; charset=utf-8')                #
    print('')                                                       #
    print("debug  : " + mmdd)                                       #
    print("comment: " + row['comment'])                             # output: comment field
    print("calc   : " + row['calc'])                                #
    print("src    : " + imgSrc)                                     # output: IMG src field
    print("text   : " + allText)                                    # output: text fields

# ==============================================================
# main
#
# sample URLs:
# Dec-25) http://neilrieck.net/cgi-bin/daily_pix?mmdd=1225&xml=1
# Jan-01) http://neilrieck.net/cgi-bin/daily_pix?mmdd=0101&xml=1
# ==============================================================
#
#   locate named data in cgi
#
def getFld(cfs, label, dflt):
    if label not in cfs:
        return dflt
    else:
        return cfs.getvalue(label)

def main():
    cfs = cgi.FieldStorage()                                        # convert CGI data into a Python data
    xml = getFld(cfs, 'xml', '0')                                   # default to "0" if missing
    json = getFld(cfs, 'json', '0')                                 #
    slot = getFld(cfs, 'slot', '0')                                 # slot may be useful when caching an image
    mmdd = getFld(cfs, 'mmdd', '0212')                              # default to Charles Darwin's birthday
    try:
        x = int(mmdd)                                               #
        if (x < 0):                                                 #
            mmdd = "0000"                                           #
        if (x > 9999):                                              #
            mmdd = "9999"                                           #
    except Exception:                                               #
        mmdd = "0212"                                               #
    #
    #   start by logging this event (default privs may not allow this)
    #   caveat: if SELinux is enabled then we may need to write to /var/log
    #
    try:
        snap = datetime.datetime.now()                              # snapshot of the current time
        date = snap.strftime('%Y%m%d')                              #
        fn = f"dp-log/dp_{date}.txt"                                # all logs go into a daily file
        f = open(fn, "a")                                           #
        dt = snap.strftime('%Y%m%d%H%M%S')                          #
        f.write(f"time: {dt}\r\n")                                  #
        f.write(f"mmdd: {mmdd} xml: {xml} json: {json})\r\n")       #
        f.close()                                                   #
    except Exception as e:                                          #
        msg = f"error: {e} in logger"                               #
        logIt(msg)                                                  #
        pass                                                        #
    #
    row = get_record(mmdd)                                          # fetch one record from our database
    #
    # return data in the requested format
    #
    if (json == "1"):
        output_json(row, mmdd, slot)                                # output JSON data
    elif (xml == "1"):
        output_xml(row, mmdd, slot)                                 # output xml data
    else:
        output_plain(row, mmdd, slot)                               # output plain data

#
#   catch-all
#
if __name__ == '__main__':
    main()
#

External Links


Back to Home
 Neil Rieck
 Waterloo, Ontario, Canada.