Archives For Gocode

micro wiki written in go

January 21, 2018

It’s been quite a while since I last blogged about writing in Google’s Go. The language itself has evolved considerably over that period of time, but my skills in it have basically stood still. Go, however, has not, especially its use around today’s contemporary software environments, such as containers. It’s also available as a regular installable package on Arch ARM for the Raspberry Pi 2/3.

I’ve been investigating network programming on the Raspberry Pi, looking for a minimalist set of tools that can provide robust connectivity and functionality. One of the tools I’ve been looking for is a web server or wiki. Turns out that there’s a Go tutorial that leads the reader through the steps necessary to create a minimal wiki. It’s located here: Writing Web Applications.

Once I finished the tutorial I followed through with the suggested simple tasks at the bottom of the tutorial. One of them was the ability to add WiKi markup to create tags within the body of any page I created. I followed Help:Wikitext for syntax and to see how regular Wiki handles markup. I implemented very little: links, headers, single unordered lists, italic and bold text. I felt that was enough to get started, and it’s enough to drop onto a Raspberry Pi as a bare executable. There are a few more key features I’d like to add, such as table support, but that’s for another round of coding. Now, I’m taking a break while I go off and do other tasks.

Here’s the code.

// Personal Wiki Sandbox.
//
// Original Copyright 2010 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//

package main

import (
	"html/template"
	"io/ioutil"
	"net/http"
	"os"
	"regexp"
	"fmt"
)

import s "strings"
var pr = fmt.Println

const dataDir = "data/"
const dataType = ".txt"
const templateDir = "templates/"

type Page struct {
	Title       string
	Body        []byte
	DisplayBody template.HTML
}

func (page *Page) save() error {
	os.Mkdir(dataDir, 0775)
	filename := dataDir + page.Title + dataType
	return ioutil.WriteFile(filename, page.Body, 0600)
}

func loadPage(title string) (*Page, error) {
	filename := dataDir + title + dataType
	body, err := ioutil.ReadFile(filename)
	if err != nil {
		return nil, err
	}
	return &Page{Title: title, Body: body}, nil
}

// I'm a regular expression neophyte. If you can figure out what I'm doing and can offer a better way of writing
// any of them, then I'm ready to learn.
//
var linkRegexp =   regexp.MustCompile( "\\[\\[([\\w ]+)\\]\\]" )
var headingOne =   regexp.MustCompile( "^=(.*)=$" )
var headingTwo =   regexp.MustCompile( "^==(.*)==$" )
var headingThree = regexp.MustCompile( "^===(.*)===$" )
var headingFour =  regexp.MustCompile( "^====(.*)====$" )
var headingFive =  regexp.MustCompile( "^=====(.*)=====$" )
var headingSix =   regexp.MustCompile( "^======(.*)======$" )
var boldText =     regexp.MustCompile( "'''(.*?)'''" )
var italicText =   regexp.MustCompile( "''(.*?)''" )

func expandHeading(line string) string {
    result := line

    if headingSix.MatchString(line) {
        result = "<h6>" + s.TrimSpace(s.Trim(line, "=")) + "</h6>\n"
    } else if headingFive.MatchString(line) {
        result = "<h5>" + s.TrimSpace(s.Trim(line, "=")) + "</h5>\n"
    } else if headingFour.MatchString(line) {
        result = "<h4>" + s.TrimSpace(s.Trim(line, "=")) + "</h4>\n"
    } else if headingThree.MatchString(line) {
        result = "<h3>" + s.TrimSpace(s.Trim(line, "=")) + "</h3>\n"
    } else if headingTwo.MatchString(line) {
        result = "<h2 style=\"margin-bottom: 0.25em; border-bottom: 1px solid #808080;\">" +
            s.TrimSpace(s.Trim(line, "=")) + "</h2>\n"
    } else if headingOne.MatchString(line) {
        result = "<h1 style=\"margin-bottom: 0.25em; border-bottom: 2px solid #808080;\">" +
            s.TrimSpace(s.Trim(line, "=")) + "</h1>\n"
    }

    return result
}

func viewHandler(writer http.ResponseWriter, request *http.Request, title string) {
	page, err := loadPage(title)

	// If the page doesn't exist, then redirect to the editing function and create the page.
	//
	if err != nil {
		http.Redirect(writer, request, "/edit/"+title, http.StatusFound)
		return
	}

	// Start page processing.
	// Break on the new line (\n) because that's common to both Windows/DOS and Unix and Unix-like editors.
	// We'll strip off any carriage returns (\r) as white space.
	//
	lines := s.Split(string(page.Body), "\n")

	var buildingUnorderedList bool
	// expandedBody is where all the displayable text with HTML decorations will be collected.
	//
	var expandedBody []string

	for _, oneline := range lines {
		//
		// Strip all leading and trailing white space, including carriage returns (\r).
		// This will help later when we join all the strings back together using the HTML break token
		// followed by a new line (\n).
		//
		newLine := s.TrimSpace(oneline)

        // Bold and italicize any marked text. Try to handle the mixing of both.
        //
        newLine = boldText.ReplaceAllStringFunc(
            newLine,
            func(str string ) string {
                matched := boldText.FindStringSubmatch(str)
                return "<b>" + matched[1] + "</b>"
            })

        newLine = italicText.ReplaceAllStringFunc(
            newLine,
            func(str string ) string {
                matched := italicText.FindStringSubmatch(str)
                return "<i>" + matched[1] + "</i>"
            })

        // Create links from [[PageName]] pattern.
        // Note that we can accept a link with spaces. I substitute the '_' for a space. I could, perhaps,
        // use the %20 called for in the standards, but I decided underscores were better.
        //
        newLine = linkRegexp.ReplaceAllStringFunc(
            newLine,
            func(str string) string {
                matched := linkRegexp.FindStringSubmatch(str)
                return "<a href=\"/view/" + s.Replace(matched[1], " ", "_", -1) + "\">" + matched[1] + "</a>"
            })

        // newLine = expandHeading(newLine)

		// Look for unordered lists. If we find an unordered list notation, then start building that list.
		//
		if s.HasPrefix(newLine, "*") {
			nl2 := s.TrimSpace(s.SplitAfterN(newLine, "*", 2)[1])

			if !buildingUnorderedList {
				buildingUnorderedList = true
				expandedBody = append(expandedBody, "<ul>\n")
			}

			expandedBody = append(expandedBody, "<li>"+nl2+"</li>\n")
			continue
		} else if buildingUnorderedList {
            buildingUnorderedList = false
            expandedBody = append(expandedBody, "</ul>\n")
		}

		// Look for headings and the rest of the document.
		//
		if s.HasPrefix( newLine, "=" ) && s.HasSuffix( newLine, "=" ) {
            expandedBody = append(expandedBody, expandHeading(newLine))
            continue
        }

        expandedBody = append(expandedBody, newLine+"<br />\n")
	}

	// Rejoin all the lines we created with the initial split using he HTML break followed by a new line.
	//
	page.DisplayBody = template.HTML([]byte(s.Join(expandedBody, "")))
	renderTemplate(writer, "view", page)
}

func editHandler(writer http.ResponseWriter, request *http.Request, title string) {
	p, err := loadPage(title)
	if err != nil {
		p = &Page{Title: title}
	}
	renderTemplate(writer, "edit", p)
}

func saveHandler(writer http.ResponseWriter, request *http.Request, title string) {
	body := request.FormValue("body")
	p := &Page{Title: title, Body: []byte(body)}
	err := p.save()
	if err != nil {
		http.Error(writer, err.Error(), http.StatusInternalServerError)
		return
	}
	http.Redirect(writer, request, "/view/"+title, http.StatusFound)
}

var templates = template.Must(
	template.ParseFiles(templateDir+"edit.html", templateDir+"view.html"))

func renderTemplate(writer http.ResponseWriter, tmpl string, p *Page) {
	err := templates.ExecuteTemplate(writer, tmpl+".html", p)
	if err != nil {
		http.Error(writer, err.Error(), http.StatusInternalServerError)
	}
}

func rootHandler(writer http.ResponseWriter, request *http.Request) {
	http.Redirect(writer, request, "/view/FrontPage", http.StatusFound)
}

var validPath = regexp.MustCompile("^/(edit|save|view)/([\\w ]+)$")

func makeHandler(fn func(http.ResponseWriter, *http.Request, string)) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		m := validPath.FindStringSubmatch(r.URL.Path)
		if m == nil {
			http.NotFound(w, r)
			return
		}
		fn(w, r, m[2])
	}
}

func main() {
	http.HandleFunc("/", rootHandler)
	http.HandleFunc("/view/", makeHandler(viewHandler))
	http.HandleFunc("/edit/", makeHandler(editHandler))
	http.HandleFunc("/save/", makeHandler(saveHandler))

	http.ListenAndServe(":8080", nil)
}

Here’s a typical markup page.

= Main Page =

This is the front page. Basic testing of all the capabilities that can be added to a page of text with minimal Wiki markup support.

= Heading 1 With ''Bloody === Equals'' In The Middle =
== Heading 2 ==
=== Heading 3 ===
==== Heading 4 ====
===== Heading 5 =====
====== Heading 6 ======

Example testing of links.

[[TestName1]] [[foobar]]
[[TestName2]]
[[Test Name 3]]

[[ANewPage]]
[[ANewPage2]]
* one ''italic'' bulleted list entry with a link: [[ANewPage]]
* two '''bold''' bulleted list entry
Example of testing '''bold text multiple''' times in the '''same''' sentence.
Example of testing ''italic '''text''' multiple'' times in the ''same'' sentence.
This is ''' ''Italic and Bold text together.'' '''

And here’s what it looks like rendered in Chrome on a Macbook Pro.

Features

  1. Handles links with [[ and ]] notation. Also handles spaces within a page link.
  2. Handles italic and bold text, and combinations of the two.
  3. Handles headings, from h1 to h6.

Blank lines and lines that aren’t headings and part of an unordered list have a single HTML break appended to them. Other than that it’s just a way to organize text. I’m even writing a stripped down version that won’t allow you to edit any of the pages, for embedded read-only rendering.

I spent a bit of time today getting my various Raspberry Pi boards up to snuff. I worked with one Raspberry Pi 2 Model B V1.1 (in the foreground) and my newer Raspberry Pi 3 Model B V1.2 (background). Both boards are back to using Arch Linux ARM, my comments about going off on a tangent with Raspbian not withstanding. I’m using the RPi 3 as a “build system”, transferring binaries to the RPi 2. Specifically I’m writing code with Google’s Go (golang). The compilation of Go code on the RPi 3 is quite fast. Test Go binaries are secure copied (scp) from the RPi3 to the RPi2 for testing.

I’d written earlier on this blog about trying to use Go on the older Raspberry Pi B+, back when Go was around version 1.4, and back when there were no installable packages for Arch Linux ARM. Because of Go’s lack of full native support on ARM at that time I went off and used node.js and the onoff package to write code in Javascript to manipulate the GPIO pins. That experimentation proved quite successful. But I wanted the same capabilities using Go if at all possible for one very important reason: Go compiles down to a single binary that doesn’t need a special runtime and supporting packages to be installed.

Recently I came across Embd, a Golang-based framework for manipulating all of a Raspberry Pi’s peripherals. It also claims to perform the same way on the Beagle Bone Black, another strong attractor for me as I have a BBB as well. I installed Go V1.6.2 (the latest release) using the standard Arch Linux ARMv7 packages. git was also installed on the RPi3, so I grabbed a copy of embd on Github (go get github.com/kidoman/embd). Once that was accomplished I started to write simple Go applications and tried to see how it operated. And I ran into more than a few problems getting it to work.

An example of a problem I ran into, and my solution to prove that at least a part of embd works, was the sample application fullblinker.go. It’s a test application that’s supposed to toggle the green LED, next to the red power LED, on the corner of the RPi board. As originally delivered it turned the LED on, but it was a solid light, not blinking. After rummaging around a bit in the source, I modified fullblinker to actually blink the LED on the RPI3, then copied it over to the RPI2 and tested it successfully as well. The photo above shows the on state of the LED during the blink test.

// +build ignore

// LED blinker, works OOTB on a RPi.

package main

import (
	"flag"
	"os"
	"os/signal"
	"time"

	"github.com/kidoman/embd"
	_ "github.com/kidoman/embd/host/rpi"
)

func main() {
	flag.Parse()

	if err := embd.InitLED(); err != nil {
		panic(err)
	}
	defer embd.CloseLED()

	led, err := embd.NewLED(0)
	if err != nil {
		panic(err)
	}
	defer func() {
		led.Off()
		led.Close()
	}()

	quit := make(chan os.Signal, 1)
	signal.Notify(quit, os.Interrupt, os.Kill)
	defer signal.Stop(quit)

    var isOff   bool = true

	for {
		select {
		case <-time.After(250 * time.Millisecond):
            if isOff {
                isOff = false
                if err := led.On(); err != nil {
				    panic(err)
			    }
            } else {
                isOff = true
                if err := led.Off(); err != nil {
				    panic(err)
			    }
            }

		case <-quit:
			return
		}
	}
}

The changes I made to the code are highlighted above. I had to replace the Toggle() function with logic that maintained the LED on or off state, and explicitly call the On() and Off() functions (highlighted lines 43 to 53). Once flashing, this at least validated part of embd. One requirement for running fullblinker is the need to run with sudo (as root), which I’d managed to eliminate with the onoff nodejs GPIO application by using a special group with udev rules. Which appears to cause additional complications with embd, as none of the embd GPIO examples work, failing with a “write /sys/class/gpio/export: device or resource busy” error message. I’ve yet to dig into that problem, but I have several ideas on how to proceed to debug the issue.

embd is a start, along with node.js and onoff. I much prefer how the Javascript application was easy to set up, and I’m mulling over the idea of replicating the Javascript API in spirit to an equivalent GO framework, using what I can pick up from embd.

At this point you’re probably asking why bother? It’s so boring, so mundane. When Google talks about it’s AI TPU, or the net chatters about the rivalry between VR platforms, writing at this low a level on a $35 computer seems like such a waste of time. I can assure you it isn’t. Sooner or later you get back down to this level, and if it doesn’t work at this level then everything further up the stack goes to shit. One major driver to my working on this is my deep appreciation of the Go language, and my desire to use it here. Once I get it working to my satisfaction I’ll have tools that I can build upon, and perhaps something I can share back with the community. I’m just a regular engineer, not a disrupter, and as the Guardian once wrote, it’s the regular people who keep the world afloat.