Python Notes: Picture-of-the-Day

  1. The information presented here is intended for educational use.
  2. The information presented here is provided free of charge, as-is, with no warranty of any kind.
Edit: 2020-07-11 (renovated my python program)

Overview

  • my original picture-of-the-day code (see the top right corner of this home page: http://neilrieck.net/) was written entirely in JavaScript. When you passed it a month-day pair in "mmdd" format, it would provide the URL (either local or remote) of one image along with some displayable text and an optional hyperlink to some article.
  • Adding support for more images was a pain so I moved all the logic into a Python program while the data was moved into a MySQL/MariaDB database. The whole thing is designed to return "escaped data" in XML format. Click the next hyperlink to see a simple example for "0212" (Feb-12) which celebrates the birthday of Charles Darwin
    xml http://neilrieck.net/cgi-bin/daily_pix?xml=1&mmdd=0212
    json http://neilrieck.net/cgi-bin/daily_pix?json=1&mmdd=0212
    plain text http://neilrieck.net/cgi-bin/daily_pix?mmdd=0212
  • this new Python application is composed of four files
    1. one html file (a web page to test my code; can cycle from 0101 to 1231)
    2. one java script file (only run client-side)
    3. one bash script (launches my server-side program)
    4. one python file (only run server-side)
    5. database schema
      database: hack1 table: pixofday
      field name use
      mmdd0 first month-day to show a picture
      mmdd1 last month-day to show a picture
      comment text used in debugging
      src local or remote location of the desired image; insert this into an IMG tag
      calc if not blank, then do an anniversary calculation; eg. 2019 was the fiftieth anniversary of the Apollo-11 landing in 1969
      text1 preformatted html
      text2 preformatted html (optional)
      text3 preformatted html (optional)
    6. database other:
      1. be sure to enter "0000" and "9999" to ensure that a default is always returned
      2. one of my special entries lays between "8000" and "8099"
  • clicking a button on the web page calls a JavaScript routine
    • one JavaScript routine, named doReset(), works entirely client-side without annoying the server
    • two JavaScript routines, named setDefaults1() and setDefaults2(), will call the server (via AJAX) expecting some sort of form initialization
    • one JavaScript routine, named runDemo(), reads all the fields defined in the HTML form, tests them, then passes their contents on to the server (via AJAX) for processing
  • because this is a simple how-to demo, many of the routines are simplistic (e.g. no reliance on JavaScript libraries: jQuery, AngularJS, ReactJS although I was tempted to do so since my code would have been much smaller)
  • there are numerous ways to pass data back to HTML but here I used XML because all modern browsers contain routines to perform XML parsing
  • I also added JSON support since it has been available with JavaScript-1.5 (ECMAscript-5)

file-1: HTML web page

<!DOCTYPE html>
<html lang="en">
<head>
 <meta content="text/html; charset=utf-8" http-equiv="Content-Type" />
 <meta content="Neil S. Rieck (Waterloo, Ontario, Canada)" name="description" />
 <meta name="viewport" content="width=device-width, initial-scale=1">
 <meta content="#ffffff" name="theme-color">
 <title>Test Pix</title>
 <link href="css/nsr-20170909.css" rel="stylesheet" type="text/css">
 <script src="js/picture-viewer4.js"></script>
</head>
<body lang="en-ca" >
 <h4>Test-Pix</h4>
 <table class="tbl-blue" style="width:100%">
  <tr><td>msg</td><td id="msg"></td></tr>
  <tr><td>src</td><td id="src"></td></tr>
  <tr><td style="width:50px">com</td><td id="comment" style="min-width:90%"></td></tr>
  <tr><td>txt</td><td id="text" style="text-align: center; font-weight: bold;height:280px"></td></tr>
  <tr><td>calc</td><td id="calc"></td></tr>
  <tr><td>mmdd</td><td id="mmdd"></td></tr>
 </table>
<script>
 testYear();
</script>
</body>
</html>

file-2: JavaScript

// =====================================
// title  : picture-viewer4.js
// author : Neil Rieck
//	    Waterloo, Ontario, Canada
// created: 2019-11-28
// =====================================
//
//	preloading images helps to reduce jitter
//
function preloadImage(url) {
	try{
		var	_img = new Image();
			_img.src = url;
	}catch(e){ }
}
//
//	my new site always displays this image first so preload in now
//
preloadImage("https://www.esrl.noaa.gov/gmd/webdata/ccgg/trends/co2_data_mlo.png");
//
//	need some global variables (ugh!)
//
var state=0;		// ajax state
var timeOutId1=0;	// ajax timer ID
var debugFlag=0;	//
var mySrc=["",""];	// image source
var myTxt=["",""];	// payload
var slot=0;		//
//
//	start_ajax (start an AJAX transaction)
//
function start_ajax(msg){
    response=null;
    if (typeof XMLHttpRequest == "undefined")
		XMLHttpRequest=function(){
		try { return new ActiveXObject("Msxml2.XMLHTTP.6.0")    } catch (e) {}
		try { return new ActiveXObject("Msxml2.XMLHTTP.3.0")    } catch (e) {}
		try { return new ActiveXObject("Msxml2.XMLHTTP")        } catch (e) {}
		try { return new ActiveXObject("Microsoft.XMLHTTP")     } catch (e) {}
		throw new Error("This browser does not support XMLHttpRequest or XMLHTTP.")
    }
    if (msg != ""){
		response=new XMLHttpRequest();
		if (response != null){
		    response.onreadystatechange=ajax_event_handler;
		    response.open("GET", msg, true);	// async=true
		    response.send(null);		// not null for POST
		}
		if (debugFlag>0) document.getElementById("msg").innerHTML="Busy";
		state = 1;
		init_timer1();
    }
}
//
//	ajax_event_handler (only executed if something is received) 
//
function ajax_event_handler(){
	if (response.readyState == 4){
		state=0;
		if (timeOutId1!=0){
			clearTimeout(timeOutId1);
	    	timeOutId1=0;
		}
		if (response.status == 200){
			if (debugFlag>0)
				document.getElementById("msg").innerHTML="Ready";
			var resp$=response.responseText;
			if (debugFlag>=2){
				console.log("raw data:"+htmlEncode(resp$));
			}
			if (window.DOMParser) {
				parser=new DOMParser();
				gXmlDoc=parser.parseFromString(resp$,"text/xml");
			}else{
				gXmlDoc=new ActiveXObject("Microsoft.XMLDOM");
				gXmlDoc.async=false;
				gXmlDoc.loadXML(resp$);
			}
			var src	= get_xml_data("src");
			var text= get_xml_data("text");
			var slot= get_xml_data("slot");
			try{
				slot=parseInt(slot);
				mySrc[slot]=src;
				myTxt[slot]=text;
			}catch(e){}
			console.log("src:"+src);
			console.log("text:"+text);
			if (debugFlag>0){
				var calc	= get_xml_data("calc");
				var comment	= get_xml_data("comment");
				console.log("calc:"+calc);
				console.log("comment:"+comment);
				document.getElementById("calc").innerHTML=calc;
				document.getElementById("comment").innerHTML=comment;
				document.getElementById("src").innerHTML=src;
				document.getElementById("text").innerHTML=text;
			}
		}
	}
}
//
//	the server must answer back in 3 seconds
//
function init_timer1(){
    if (timeOutId1==0){						// if available
		timeOutId1=setTimeout("timer_job1();",3000);	// arm for 3 seconds
    }
}
//
//	this code is ONLY executed on TIMEOUT
//
function timer_job1(){
    state=0;						// reset state
    if (timeOutId1!=0){
		clearTimeout(timeOutId1);
		timeOutId1=0;
    }
    var txt="Error: the timer has expired";
    if (debugFlag>0)	document.getElementById("msg").innerHTML=txt;
}
//
//	does string 'x' represent a positive integer?
//
function isStrInt(x){
	x=x.replace(/\s/g,'');
    if (x=="") return false;
    try{
		y = parseInt(x);
    }catch(e){
		y = -1;
    }
    if(y>0){
    	return true;
    }else{
		return false;
    }
}
function htmlEncode(str) {
    return String(str).replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;').replace(/"/g, '&quot;');
}
function htmlDecode(str) {
    return String(str).replace(/&lt;/g, '<').replace(/>&gt;/g, '>').replace(/&quot;/g, '"').replace(/&amp;/g, '&');
}
//
//	get_xml_data (look for one item)
//
function get_xml_data(tag){
    var x;
    try{
	x = gXmlDoc.getElementsByTagName(tag)[0].childNodes[0].nodeValue;
    }catch(e){
	console.log("get_xml_data tag:"+tag+" err:"+e);
	x = null;
    }
    return(x);
}
//
//	get_xml_data2 (look for numerous similar items)
//
function get_xml_data2(xmlObj, tag){
    var x;
    try{
	x = xmlObj.getElementsByTagName(tag)[0].childNodes[0].nodeValue;
    }catch(e){
	console.log("get_xml_data2 tag:"+tag+" e:"+e);
        x = null;
    }
    return(x);
}
//
//	get number of days per month
//
function daysInMonth (month, year){
	return new Date(year, month, 0).getDate();
}
//
//	left-pad with spaces
//
function pad(num, size) {
	var s = num+"";
	while (s.length < size) s = "0" + s;
	return s;
}
//
//	test picture-of-the-day logic for the whole year
//	caveat: only used in testing
//
function testYear(){
	debugFlag=1;		// change global
	var mm=0;		// month
	var dd=0;		// day
	var mmdd="";
	var intervalId = setInterval(function(){
		if (mm==0){
			mm=1;
			dd=1;
		}else{
			dd++;
			limit=daysInMonth(mm);
			if (dd>limit){
				mm++;
				dd=1;
			}
		}
		if (mm>12){
			clearInterval(intervalId);
		}else{
			mmdd=junk=pad(mm,2)+pad(dd,2);
			console.log("mmdd:"+mmdd);
			document.getElementById("mmdd").innerHTML="mmdd: "+mmdd;
			msg = "http://neilrieck.net/cgi-bin/daily_pix?xml=1&slot=0&mmdd="+mmdd;
			start_ajax(msg);
		}		
	}, 1000);
}
//
//	get one picture for today
//
function fetchToday(){
	debugFlag=0;
	slot=1;
	var d	= new Date();
	var mm	= d.getMonth();	// 0..11
	var dd  = d.getDate();	// 1..31
	var mmdd= pad(mm,2)+pad(dd,2);
	console.log("mmdd:"+mmdd);
	msg = "http://neilrieck.net/cgi-bin/daily_pix?xml=1&slot=0&mmdd="+mmdd;
	start_ajax(msg);
}
//
//	get default picture
//
function fetchDefault(){
	debugFlag=0;
	slot=0;
	var mmdd="8000";
	console.log("mmdd:"+mmdd);
	msg = "http://neilrieck.net/cgi-bin/daily_pix?xml=1&slot=1&mmdd="+mmdd;
	start_ajax(msg);
}

file-3: Python3 (optional launcher)

note: this file is usually located in the cgi-bin directory of your web server
 
#!/usr/bin/python3
# title  : daily_pix
# author : Neil Rieck
# edit   : 2019-11-23
# notes  : this is just a loader
# ------------------------------
import daily_pix_131.py
daily_pix_131.main()

file-4: Python3 (program)

note: in this example, this file is found under cgi-bin (but you could move it anywhere else provided Apache can access it)
#!/usr/bin/python3
'''
===================================================================================
title  : daily_pix_131.py
author : Neil Rieck
history:
2019-11-23 daily_pix_main_100.py - calls daily_pix_mysql.py
2019-12-15 daily_pix_main_101.py - cleanup
2020-04-18 daily_pix_main_102.py - added JSON support
2020-05-09 daily_pix_130.py      - moved db here; cursor now returns a dictionary
2020-07-11                       - tiny fix in anniversary calc
2020-07-11 daily_pix_131.py      - replaced format strings with f strings
purpose:
1) for a given month-day (format: mmdd) return a picture along with associated text
2) optionally, return the data in XML or JSON format
3) these PEP8 rules are disabled: E305,E501 so I can code in a traditional way
===================================================================================
'''
import cgi
import datetime
import json
import sys
#   sudo python3 -m pip install mysql-connector
import mysql.connector as db
#
hostUrl = "http://neilrieck.net"                                    # default URL of this site (read on)
#
#   common function to BAIL (will be visible in the browser's console)
#
def errorExit(msg):
    print("content-type: text/plain; charset=utf-8")
    print("")
    print(msg)
    sys.exit()
#
#   logIt (a simple file-based logger)
#   caveat: if SELinux is enabled then write to /var/log/daily_pix_msg_log.txt
#
def logIt(msg):
    try:
        fn = "daily_pix_msg_log.txt"
        f = open(fn, "a")
        f.write(f"msg: {msg}\r\n")
        f.close()
    except Exception:
        pass
#
#   do that database thing
#
def get_record(parm):
    try:
        con = db.connect(host='127.0.0.1', user='read123', passwd='read123', database='hack1')
        cur = con.cursor(dictionary=True)                           # force return of dict
        cmd = "select * from pixofday "
        cmd += f"where (mmdd1 <= {parm}) and (mmdd2 >= {parm}) "
        cmd += "order by concat(mmdd2,mmdd1) limit 1"
        cur.execute(cmd)
        # row = cur.fetchall()                                      # returns a list of tuples (or dicts)
        row = cur.fetchone()                                        # returns a tuple (or dict)
        return row
    except Exception as e:
        msg = f"Exception: {e} while opening database"
        logIt(msg)
        errorExit(msg)

'''
===============================================================

 <<< general description >>>

 1) database field definitions
   0: mmdd1 (start date)
   1: mmdd2 (end date)
   2: comment
   3: calc  (if not blank then do an anniversary calculation)
   4: src   (image source)
   5: text1 (usually a title associated with the photo)
   6: text2
   7: text3
 2) data:
   1: each record will indicate that a certain photo be shown between mmmdd1 and mmdd2
   2: one special record has mmdd1="0000" and mmdd2="9999" so something will always be returned
 3) logic:
   1: the 'src' field contains an image location like these examples:
      /images/Edmond_Halley.jpg
      https://www.esrl.noaa.gov/gmd/webdata/ccgg/trends/co2_data_mlo.png
   2: since a couple of my websites are using this logic, any entry not containing "http" will need to be
      prefixed with the URL of this site
   3: text1 to text3 contain HTML so merge them into one string then test for the following patterns:
   4: if we detect the string "+xxx+" then replace it with the resultant 'src' string seen at the end of step 2
   5: if we detect the string "+host+" then replace it with the URL of this site
   6: if we detect the string "+calc+" then do an anniversary calculation (eg. 50th anniversary of Apollo-11)
      then replace the target string with the result
   7: return data to caller via XML or JSON (this means the data must be properly escaped)
 4) Cross-Origin Resource Sharing (CORS)
    For AJAX use you must add a few Cross-Origin Resource Sharing (CORS) directives to Apache config like so:
    <Directory "/var/www/html/cgi-bin">
      AllowOverride None
      Options None
      Require all granted
      # 1) CORS for initial AJAX testing.
      Header set Access-Control-Allow-Origin "*"
      # 2) CORS for single second site support (this Apache instance runs on "http://neilrieck.net"
      # but we want to allow AJAX access from "http://www3.sympatico.ca/n.rieck/")
      #Header set Access-Control-Allow-Origin http://www3.sympatico.ca
      #Header set Access-Control-Allow-Headers "Content-Type,Accept"
      #Header merge Vary Origin
      #Header set MyHeader "Neil's Hack"
     </Directory>
===============================================================
'''
def process_data(row):
    imgSrc = row['src']                                             # logic-1
    if (imgSrc.find('http') == -1):                                 # logic-2
        imgSrc = hostUrl + imgSrc                                   #
    allText = row['text1'] + row['text2'] +row['text3']             # logic-3
    temp = allText.find('+xxx+')                                    # logic-4
    if (temp != -1):                                                # if found...
        allText = allText[:temp] + imgSrc + allText[temp+5:]        #
    temp = allText.find('+host+')                                   # logic-5
    if (temp != -1):                                                # if found...
        allText = allText[:temp] + hostUrl + allText[temp+6:]       #
    temp = allText.find('+calc+')                                   # logic-6
    if (temp != -1):                                                # if found...
        year = datetime.datetime.now().strftime('%Y')               #
        try:                                                        #
            anniv = int(row['calc'])                                # eg. ccyy
        except Exception as e:                                      #
            anniv = 1900                                            #
            msg = f"error: {e}"                                     #
            logIt(msg)                                              #
        diff = int(year) - anniv                                    #
        allText = allText[:temp] + str(diff) + allText[temp+6:]     #
    return allText, imgSrc                                          #

def output_xml(row, mmdd, slot):
    allText, imgSrc = process_data(row)                             #
    escText = cgi.escape(allText, allText)                          # logic-7
    print('Content-Type: text/xml; charset=utf-8')                  # we're returning an XML response
    print('')                                                       # end of HTTP header
    print('<?xml version="1.0" encoding="utf-8"?>')                 # start of XML content
    print('<result>')                                               # user-defined by me (see JavaScript)
    print('<status>1</status>')                                     #
    print('<debug>' + mmdd + '</debug>')                            # the parameter passed in
    print('<comment>' + row['comment'] + '</comment>')              #
    print('<calc>' + row['calc'] + '</calc>')                       #
    print('<src>' + imgSrc + '</src>')                              #
    print('<text>' + escText + '</text>')                           #
    print('<slot>' + slot + '</slot>')                              #
    print('</result>')                                              #
#
def output_json(row, mmdd, slot):
    allText, imgSrc = process_data(row)
    print('Content-Type: text/json; charset=utf-8')                 # we're returning an XML response
    print('')                                                       # end of HTTP header
    del row['text3']                                                # enabled new logic
    del row['text2']
    del row['text1']
    row['imgSrc'] = imgSrc                                          # append
    row['text'] = allText                                           # append
    print(json.dumps(row, indent=1))
#
def output_plain(row, mmdd, slot):                                  #
    allText, imgSrc = process_data(row)                             #
    print('Content-Type: text/plain; charset=utf-8')                #
    print('')                                                       #
    print("debug  : " + mmdd)                                       #
    print("comment: " + row['comment'])                             # output: comment field
    print("calc   : " + row['calc'])                                #
    print("src    : " + imgSrc)                                     # output: IMG src field
    print("text   : " + allText)                                    # output: text fields

# ==============================================================
# main
#
# sample URLs:
# Dec-25) http://neilrieck.net/cgi-bin/daily_pix?mmdd=1225&xml=1
# Jan-01) http://neilrieck.net/cgi-bin/daily_pix?mmdd=0101&xml=1
# ==============================================================
#
#   locate named data in cgi
#
def getFld(cfs, label, dflt):
    if label not in cfs:
        return dflt
    else:
        return cfs.getvalue(label)

def main():
    cfs = cgi.FieldStorage()                                        # convert CGI data into a Python data
    xml = getFld(cfs, 'xml', '0')                                   # default to "0" if missing
    json = getFld(cfs, 'json', '0')                                 #
    slot = getFld(cfs, 'slot', '0')                                 # slot may be useful when caching an image
    mmdd = getFld(cfs, 'mmdd', '0212')                              # default to Charles Darwin's birthday
    try:
        x = int(mmdd)                                               #
        if (x < 0):                                                 #
            mmdd = "0000"                                           #
        if (x > 9999):                                              #
            mmdd = "9999"                                           #
    except Exception:                                               #
        mmdd = "0212"                                               #
    #
    #   start by logging this event (default privs may not allow this)
    #   caveat: if SELinux is enabled then we may need to write to /var/log
    #
    try:
        snap = datetime.datetime.now()                              # snapshot of the current time
        date = snap.strftime('%Y%m%d')                              #
        fn = f"dp-log/dp_{date}.txt"                                # all logs go into a daily file
        f = open(fn, "a")                                           #
        dt = snap.strftime('%Y%m%d%H%M%S')                          #
        f.write(f"time: {dt}\r\n")                                  #
        f.write(f"mmdd: {mmdd} xml: {xml} json: {json})\r\n")       #
        f.close()                                                   #
    except Exception as e:                                          #
        msg = f"error: {e} in logger"                               #
        logIt(msg)                                                  #
        pass                                                        #
    #
    row = get_record(mmdd)                                          # fetch one record from our database
    #
    # return data in the requested format
    #
    if (json == "1"):
        output_json(row, mmdd, slot)                                # output JSON data
    elif (xml == "1"):
        output_xml(row, mmdd, slot)                                 # output xml data
    else:
        output_plain(row, mmdd, slot)                               # output plain data

#
#   catch-all
#
if __name__ == '__main__':
    main()
#

External Links


 Back to Home
 Neil Rieck
 Waterloo, Ontario, Canada.