Making your own Saktumiva server using Ubuntu

The following are instructions for using the command-line interface to set up your own instance of Charles Li’s Saktumiva (Upama) on your own Ubuntu-type server.* You’re going to be working in /var/www a lot, so you need to use sudo for almost all commands. Unless otherwise stated, you’ll do all these commands in the /var/www directory. I’m assuming that you sit in /var/www and to download files to that location, etc., using sudo all the time. (or just becoming root for a while sudo su).

1. Install apache2 (not nginx; it probably works, but we’re doing apache2 here). 

    sudo apt-get install apache2  

   You will then have a directory /var/www/html which is where all your later files for Dokuwiki and Upama will go. Next, create a file  

   /etc/apache2/sites-available/dokuwiki.conf  

   with the contents

   <VirtualHost *:80>
       ServerAdmin <a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="b1d0d5dcd8dff1d4c9d0dcc1ddd49fd2dedc">[email protected]</a>
       DocumentRoot /var/www/html/
       ServerName your_domain_or_IP
       ErrorLog ${APACHE_LOG_DIR}/error.log
       CustomLog ${APACHE_LOG_DIR}/access.log combined
   </VirtualHost>

   Replace "your_domain_or_IP" with the public IP number of your server. (I got the above from /etc/apache2/sites-available/dokuwiki.conf and I’m not certain it is necessary.) 

2. Install php and extras:
   
   
   sudo apt install php php-xml php8.1-mbstring
   
   

3. Install Dokuwiki by following the instructions at https://www.dokuwiki.org/install. Broadly, you fetch the file dokuwiki-stable.tgz by saying

   cd /var/www sudo wget https://download.dokuwiki.org/src/dokuwiki/dokuwiki-stable.tgz

   And then you’ll unpack the tgz file into /var/www/html using the commands

   cd /var/www/html sudo tar xvf /var/www/dokuwiki-stable.tgz -C /var/www/html 

   The -C arguments above tell tar to unpack the archive to the /var/www/html directory. You should then have a directory called something like /var/www/html/dokuwiki-2024-02-06a. Rename it to just ./dokuwiki:

   sudo mv  /var/www/html/dokuwiki-2024-02-06a /var/www/html/dokuwiki

4. Now fetch the Upama plugin from Github:

   cd /var/www sudo git clone https://github.com/chchch/upama.git

5. Install Upama. Charles explains what to do at the bottom of this page: https://github.com/chchch/upama . There’s no installation as such; all you do is copy the files to the right dokuwiki directories.

   To be more explicit, do the following:

   cd /var/www (where you now have an upama directory)

   Now copy the folders and files from the upama directory into the dokuwiki directory:

   sudo cp -r /var/www/upama/dokuwiki/* /var/www/html/dokuwiki/

   The folders that get copied are /var/www/upama/dokuwiki/data -> /var/www/html/dokuwiki/data /var/www/upama/dokuwiki/lib -> /var/www/html/dokuwiki/lib  

   One last thing:

   sudo touch /var/www/html/dokuwiki/data/pages/sidebar.txt  

   This creates the empty file /var/www/html/dokuwiki/data/pages/sidebar.txt

6. Set some file permissions and ownership. This is a bit of a nightmare, and www.dokuwiki.org’s documentation is awful. But all your really need to do is this: sudo chmod -R 777 /var/www/html/dokuwiki/data/ sudo chmod -R 777 /var/www/html/dokuwiki/lib/ sudo chmod -R 777 /var/www/html/dokuwiki/conf/ sudo chown -R www-data:www-data /var/www/html/

7. Next, some necessary apache stuff I got from https://forum.dokuwiki.org/d/9893-the-datadir-pages-at-data-pages-is-not-found/4 . We’ve already done the stuff up to Step C. Now edit the file apache2.conf. I use the editor vi; pick your favourite non-graphics editor.

   sudo vi /etc/apache2/apache2.conf 

    Find the lines

   <Directory /var/www/>
   ...
   </Directory>

   and below them add the following two new sections:

   <Directory /var/www/dokuwiki>
     order deny,allow
     allow from all
   </Directory>
   
   <LocationMatch "/(data|conf|bin|inc)/">
       order allow,deny
       deny from all
       satisfy all
   </LocationMatch>

   Then restart the apache server: sudo service apache2 restart Open a new browser tab and go to the page <my_IP_address>/dokuwiki/install.php This should open a page of dokuwiki settings. This page only appears the first time you invoke it; later, you get instructions on how to change things in a more low-level manner. At your dokuwiki page, log in and go to Admin/Configuration Manager. Amongst the initial “Basic” settings, find the setting for “template” and change it from “dokuwiki” to “saktumiva”. It looks like this:

    

   And go to the bottom of the screen and hit "save". 

8. Stay with me. We’re almost done. The next thing is to add some TEI files of your edition.They are going to live in this directory:

     cd /var/www/html/dokuwiki/data/pages/wiki/  

   Create a file structure under wiki/ that makes sense to you. For example, sudo mkdir /var/www/html/dokuwiki/data/pages/wiki/Sakuntala/

   Copy your TEI files into ../Sakuntala/.  

   sudo cp /home/me/Documents/SaktutalaTEIfiles/* /var/www/html/dokuwiki/data/pages/wiki/Sakuntala

   Remember that the files need the extension .txt, even if they are xml files.

   You can also create a file start.txt in the ../Sakuntala/ directory that contains a nice starting page for your project. This must be written in dokuwiki syntax (cheatsheet).

9. At this point, you should have a usable system. However, there’s a useful tweak. If you are going to upload your TEI files from your desktop to your new Saktumiva server by ftp, perhaps using Filezilla, you will want to change the ownership of files under ../Sakuntala/ so that your ftp identity can write files to that directory. sudo chown -R <me>:<me> /var/www/html/dokuwiki/data/pages/wiki/Sakuntala/

   where “<me>” is your Filezilla ftp login username. (You can set up users in Dokuwiki under Admin/User Manager).

10. Other tweaks.
    
    • If your collation-generation times out, try upping the max execution in /etc/php8.1/apache/php.ini.
    • If your installation can't find symlinked files under directories, try adding the option +FollowSymLinks to  /etc/apache2/sites-enabled/000-default.conf:
      
      DocumentRoot /var/www/html
              <Directory />
              Options +FollowSymLinks
              AllowOverride None
              </Directory>
    • As a variant of point 9. above, you may prefer to use Saktumiva on your own laptop or desktop, rather than from a remote server.  If you want to link files from somewhere else on your disk into the Dokuwiki data/pages/wiki directory, you need to use hard links, not soft, symbolic links.  In Unix, ln <target> <name> not ln -s <target> <name>.   And if you want to be able to edit them through the Dokuwiki interface, say
      
      cd /var/www/html/dokuwiki/data/pages/wiki/Sakuntala/
      sudo chown -R <me>:www-data Sakuntala
      
      so that you are the "owner" but the "group" is still Apache.
      

-----------------------------------

*This installation guide by Christas Pontikis is very helpful: 

Dominik Wujastyk

June, August 2024

The demise of gksudo from Ubuntu-based distributions and what to do about it

I gather that there were good security-related reasons for discontinuing gksudo. But it's quite inconvenient not having it, for example when you are adding new items to the Linux Mint main menu (using menulibre, for example).

Here's what I do:

• install ssh-askpass
  
  
• add this line
  

  SUDO_ASKPASS=/usr/bin/ssh-askpass
  

  to the file

  /etc/environment

• Then, in menu or batch commands that you want to pop-up a graphical p/w prompt, say
  

  sudo -A <command>

Automounting google-drive-ocamlfuse: getting Google Drive to appear as a Linux folder

This is well documented elsewhere, but just for clarity, here are my settings, that work. I.e., the Google drive appears automatically mounted after a fresh boot.

1. Install google-drive-ocamlfuse

2. Line in /etc/fstab (all one line):

gdfuse#default  /home/dom/GoogleDriveUofA    fuse    uid=1000,gid=1000,allow_other,user,_netdev     0       0

3. Contents of executable /usr/bin/gdfuse

#!/bin/bash
su dom -l -c "google-drive-ocamlfuse -label $1 $*"
exit 0

Linux Mint 19.1 Cinnamon and display flickering

When I upgraded to Linux Mint 19.1 with Cinnamon, I encountered a problem that my screen flickered immediately after logging in.  Others encountered the same problem (here).
I found that the solution proposed there, i.e., to delete the line

CLUTTER_PAINT=disable-clipped-redraws:disable-culling

from the file

/etc/environment

worked for me.  Logging out and back in after that change gave me a stable display.

Fetching multiple files from an internet site as a batch job

Sometimes one encounters a website that displays a book or manuscript page-by-page as individual jpeg files.  But what you need for your research is to have a single PDF of the item, so that you can move about it easily, and consult it offline.

There are several quick ways of getting these images as a batch job: here's one.   

• First you have to identify the URL of one of the images.  I use Firefox, so I 
  • first bring up a page that displays the first folio of the MS. 
  • Then I press ctrl+I to get the "page info" (or Firefox menu Tools/Page Info).  
  • Then I select "Media" on the top line of the Info window.  
  • Then scroll down to the graphics file of the whole page, right click and ctrl+c to copy the URL.
    
    You now have a URL that looks like this:
    
    http://awebsite.net/uploads/manuscripts/miscellaneous/sometext/001.jpg

There may be a more direct way of getting this URL, but this is good enough for me.

The next bit is the nice bit.  Drop to the command line and use the utility "curl".  Here's the syntax ($ is my command prompt):

$ curl -O http://awebsite.net/uploads/manuscripts/miscellaneous/sometext/[001-268].jpg

• Hit "enter" and several hundred jpeg files will be transferred to your directory.  It takes a couple of minutes, depending on your bandwidth.
  The bit in square brackets, "[001-268]" is curl's syntax for "please fetch 001.jpg, 002.jpg, ... 267.jpg, and 268.jpg".  Curl is one of the few tools with this simple ability to fetch lots of different files with a single simple command.

To convert them to a single PDF, I use ImageMagick:

$ convert *.jpg Hayanaratna.pdf

and wait for ten seconds.

(I was taught about curl by Patrick McAllister - thanks Patrick!)

A quite different approach is to use wget to fetch a whole website in a single gulp.   That's what I use for GRETIL, for example, so that I have the whole archive on my hard drive.

Tweaking the Thinkpad's TrackPoint settings in Linux Mint 19

With the update to Linux Mint 19 Cinnamon, the sliders controlling the trackpoint's accelleration and speed on my Thinkpad T560 and T500 don't work any more.  I expect it will get fixed soon.  Meanwhile, as root I  can manually edit the parameters in the filesin the directory

• /sys/devices/platform/i8042/serio1/serio2

The above directory is found by

• find /sys/devices/platform/i8042 -name name | xargs grep -Fl TrackPoint | sed 's/\/input\/input[0-9]*\/name$//'

Settings are reset at reboot.  :-(

I like sensitivity 125 and speed 80 on the T560; 190 and 90, inertia 10, on the T500.

Update

I found this solution (X1 parameters) that automates everything and survives reboot :-)

Another update, March 2020

Lalufu at github said,

You can also use xinput to modify the libinput values directly:
Run xinput to show the pointers the system knows about. If you have a ThinkPad you should see something along the lines of TPPS/2 IBM TrackPoint or similar.

xinput --set-prop "TPPS/2 IBM TrackPoint" "libinput Accel Speed" -0.5

will modify the pointer speed. Play around with the value to see what you like. If you want this to survive a reboot you can stick this in .bash_profile.

Improving PDFs

I sometimes  do some processing on PDFs if I think they are important, or I want to read them more conveniently.  I was trying to explain my techniques to my students, recently, and I realized that I use a mixture of tools that are not at all obvious or easy to explain to someone not familiar with Unix.
So I'm going to write down here what I do, so that at least the information is available in one place.  I assume a general knowledge of Linux and an ability to work with command-line commands.

If I receive a PDF that is a scanned book, with 1 PDF page = one book opening, I want to chop it up so that 1 PDF page = 1 book page.

• make a working directory
• use pdftk to unpack the PDF into one file per page:
  

  > pdftk foobar.pdf burst

• I now have a directory full of one-page PDFs.  Nice.
• convert them into jpegs using pdf2jpegs, a shell script that I wrote that contains this text:
  

  #!/bin/bash
  # convert a directory full of pdfs into jpegs
  for i in *.pdf; do pdftoppm -jpeg -r 400 "$i" >"$i.jpg"; done

• I now have a directory full of jpegs, one jpeg per page.
• Start the utility scan-tailor and use it to
  
  • separate left and right pages into separate files
  • straighten the pages
  • select the text area of each page
  • create a margin around the text
  • finally, write out the resulting new pages
   
• I now have a directory (../out) full of TIFF files, one page per file, smart.
•  Combine the TIFFs into a single PDF using my shell script tiffs2pdf:
  

  #!/bin/bash
  # Create a PDF file from all the .tiff files in the current directory.
  # The argument provides the name of the output file (with .pdf appended).
  echo "Created a PDF from a directory full of .tif files"
  echo "Single argument - the filename of the output PDF (no .pdf extension)"
  tiffcp *.tif "/tmp/${1}.tiff"
  tiff2pdf "/tmp/${1}.tiff" > "${1}.pdf"
  echo "Created ${1}.pdf"
  rm "/tmp/${1}.tiff"
  echo "Removed temporary file /tmp/${1}.tiff"
  
  # thanks to http://ubuntuforums.org/showthread.php?t=155628

   
• I now have a nice PDF that has one smart page per PDF page. 
• If I want it OCRed, then I usually use Adobe Acrobat, a commercial program. But if I'm uploading to Archive.org, that isn't necessary because Archive.org does the OCR work using Abbyy.

That's all, folks!

Helpful notes on bulk downloading from Archive.org

• https://gareth.halfacree.co.uk/2013/04/bulk-downloading-collections-from-archive-org
• https://pypi.python.org/pypi/internetarchive

Much easier to work with a compiled ia rather than the python version (which I couldn't get to work anyhow).

Del latitude xinput settings

See https://askubuntu.com/questions/688270/mouse-speed-too-fast

Put the following commands in a file (foobar.sh), make the file executable (chmod +x foobar.sh), and then run it.

#!/bin/sh
xinput --set-prop "AlpsPS/2 ALPS DualPoint Stick" "Device Accel Constant Deceleration" 8
xinput --set-prop "AlpsPS/2 ALPS DualPoint Stick" "Device Accel Velocity Scaling" .8
xinput --set-prop "AlpsPS/2 ALPS DualPoint Stick" "Device Accel Adaptive Deceleration" 8


You can run this command on startup from the Startup Applications menu.

IBUS bug fix ... again (sigh!)

Further to https://cikitsa.blogspot.ca/2012/01/ibus-bug-fix.html, I found the same bug cropping up in Linux Mint 18.1, with IBUS 1.15.11.

Some applications don't like IBUS + m17n, and certain input mim files. For example, LibreOffice and JabRef.  Trying to type "ācārya" will give the result is "ācāry a". And in other strings, some letters are inverted: "is" becomes "si" and so forth.

Here's the fix.

Create a file called, say ibus-setting.sh with the following one-line content:

export IBUS_ENABLE_SYNC_MODE=0

Copy the file ibus-setting.sh to the directory /etc/profile.d/, like this:

sudo cp ibus-setting.sh /etc/profile.d

Make the file executable, like this:

sudo chmod +x /etc/profile.d/ibus-setting.sh

Logout and login again.

Phew!

This fixes the behaviour of IBUS + m17n with most applications, including LibreOffice and Java applications like JabRef.  However, some applications compiled with QT5 still have problems.  So, for example, you have to use the version of TeXStudio that is compiled with QT4, not QT5. [Update September 2018: QT5 now works fine with Ibus, so one can use the QT5 version of TeXstudio with no problem.]

GNU Freefont fonts and XeLaTeX

The problem

There's been a long-standing issue about using the Gnu Freefont fonts with XeLaTeX.  The fonts are "Free Serif", "Free Sans" "Free Mono", and each has normal, italic, bold and bold-italic versions.  

These fonts are maintained by Stevan White, who has done a lot of support and maintenance work on them.  

These fonts are of special interest to people who type Indian languages because they include nice, and rather complete Devanāgarī character sets in addition to glyphs for

• Bengali
• Gujarati
• Gurmukhi
• Oriya
• Sinhala
• Tamil
  and
• Malayalam

The Gnu Freefonts are excellent for an exceptionally wide range of scripts and languages, as well as symbols.  See the coverage chart.

At the time of writing this blog, December 2014, the release version of the fonts is 4-beta, dated May 2012.  This is the release that's distributed with TeXLive 2014, and is generally available with other programs that include or require the FreeFonts.

But the 2012 release of the FreeFonts causes problems with the current versions of XeTeX.  Basically, the Devanagari conjunct consonants in the 2012 fonts are incompatible with the current XeTeX compositing engine. (For the technical: Up to TL 2012 XeTeX used ICU; since TL 2013 it's used HarfBuzz.)

In the last couple of years, Stevan has done a great deal of work on the Devanagari parts of the FreeFonts, and he has solved these problems.  But his improvements and developments are only available in the Subversion repository.   For technically-able users, it's not hard to download and compile this pre-release version of the fonts.  But then to make sure that XeTeX calls the right version of the FreeFonts, it's also necessary to weed out the 2012 version of the fonts that's distributed with TeX Live 2014.  And that's a bit hard.  In short, things get fiddly.

Now, Norbert Preining has created a special TeX Live repository for the Subversion version of the FreeFonts.  TeX Live 2014 users can now just invoke that repo and sit back and enjoy the correct Devanagari typesetting.

New warning June 2017: 
the procedure below is no longer supported.  Don't do it.

WARNING
Be warned that the version distributed here is a development version, not meant for production. Expect severe breakage. You need to know what you are doing!
END WARNING

Here follow Norbert's instructions (as of Dec 2014).  Remember to use sudo if you have TeX Live installed system-wide.

The solution. A new TeX Live repository for the pre-release Gnu FreeFonts

Norbert says (Dec 2014):

Here we go: Please do:

tlmgr repository add http://www.tug.org/~preining/tlptexlive/ tlptexlive
tlmgr pinning add tlptexlive gnu-freefont
tlmgr install --reinstall gnu-freefont

You should see something like:  

[~] tlmgr install --reinstall gnu-freefont
...
[1/1, ??:??/??:??] reinstall: gnu-freefont @tlptexlive [12311k]
...

Note the

@tlptexlive

After that you can do  

tlmgr info gnu-freefont

and should see: 

Package installed:   Yes
revision:    3007
sizes:       src: 27157k, doc: 961k, run: 19769k
relocatable: No
collection:  collection-fontsextra

Note the

revision: 3007

which corresponds to the freefont subversion revision!!!

From now on, after the pinning action, updates for gnu-freefont will

always be pulled from tlptexlive (see man page of tlmgr).
 

Reverting the change:

In case you ever want to return to the versions as distributed in TeX Live, please do

tlmgr pinning remove tlptexlive gnu-freefont
tlmgr install --reinstall gun-freefont

Thank you, Norbert!

Linux Mint 17, Cinnamon, Firefox 31 = freeze

I've been very happy with Linux Mint 17, and the Cinnamon GUI.
Except for the occasional system freezes.  This happened usually when I was swapping between windows (alt+tab), and required a re-login.

I've spent some time looking around the web and trying to diagnose the problem.  I've found one change that seems to have cleared up the problem, and I am not aware that anyone else has mentioned it yet.  I've uninstalled Firefox (31), and replaced it with Google Chrome.  I haven't had a freeze since doing that.  Fingers crossed.

Corrupted font spacing in terminal

I had this problem, that was solved by purging pango-graphite:

sudo apt-get purge pango-graphite

From Gnome to Cinnamon

Gnome 2 and 3

Ubuntu with Gnome 2

Ubuntu with Unity

After moving to Ubuntu GNU/Linux for all my work, in 2009, I used the default interface, Gnome 2, for a while.  Later, with version 3, Gnome moved to a completely new concept, but Ubuntu forked the development and moved to Unity, so I did that too.  

Gnome 3

Gnome 2 and Unity both had their virtues and their flaws.  The six-monthly upgrade cycle ("cadence") was never as smooth as it should be, so there have often been niggles that lasted a few weeks or months.  I didn't like Unity's two different search boxes.

Ubuntu with Gnome 3

I started using the Gnome Shell, ver. 3, on Linux (Ubuntu) after seeing my friends and colleagues using it at the TeX conference in Trivandrum in 2011.  I really liked Gnome 3, but with the update from 3.6 to 3.8 and 3.10, they did some major, major things wrong, and I've finally dumped it, in favour of Cinnamon.

The biggest boo-boo in the development of Gnome from version 3.8, was fooling with the default file-manager, Nautilus.  Many people have complained about the stripping out of function, like split-screen, and that was bad enough.  So was the nonsense about shifting the menus to the panel bar (or not!).  But what hasn't got mentioned so much (at all?) is that the new Nautilus changed all the keyboard shortcuts and rearranged the shortcuts relating to the menu system.  So Alt-F didn't bring up a "File" menu any more, for  example.  Right mouse-click+R didn't begin renaming a file.  If one uses computers all day, then one's fingers get trained, and no interface designer should mess with that stuff without expecting backlash.  With Nautilus 3.8, it was like being a beginning typist again, looking at my fingers, chicken-pecking for keys.

I liked the general design model of Gnome 3, with the corner switch to the meta level for choosing programs, desktops, and so on.  Searching for lesser-used programs with a few keystrokes rather than poking hopelessly through nested menus.  Much better.  A genuine and valuable contribution to the vision of how a computer should work.

Thanks to Webupd8, I was able to work around the Nautilus problem by uninstalling it and using Nemo instead.

But things just kept going wrong.  The shell crashed too often.  On two of my machines it stopped coming up at login, and had to be started manually.  Only after a couple of weeks did I track this down to a bad file in ~/.config/gnome-session (and I'm still not 100% sure).  Frequent crashes of the gnome-control-panel and other utilities.  More and more extraordinary tweaking to make it comfortable and useable.  Finally, I've had enough.

 

Cinnamon

Ubuntu with Cinnamon

I'm in my first few days of using Cinnamon, and so far things are okay.  I'm running Cinnamon on top of Ubuntu.  It's like stepping back in time, a bit, all those menus.  But one doesn't have to use them, and with a bit of tweaking one can set things up so that actual shell behaviour is very similar to Gnome 3.6.  Nemo is there - what a relief.  Alacarte actually works, but I've dumped it in favour of Menulibre in any case.  Configuration and tweaking is much nicer.  Many useful add-ons, and although I liked the http://extensions.gnome.org system, Cinnamon handles the add-on business in a much more integrated way.  Ibus+m17n work as expected again.  In general, it's an update from Gnome 2 in the direction of Gnome 3 but not the whole way.  And it seems more stable, which is critical to getting work done.

Checklist of things to do on reinstalling Ubuntu

I'm finally moving to 64 bit Linux (since all my machines are fine with that).  My disks have my /home and root files in different partitions, so I can erase and reinstall Linux itself without touching any of "my" files.  This works flawlessly, and the new installation comes up with all my old desktop settings etc.  Since the Ubuntu Software Center's "sync" function still doesn't work properly, here are some installations and customizations that I like:

Xapian niceness

In older Linuxes, if xapian indexing is sucking up all your CPU cycles, here's a fix:

• http://ubuntuforums.org/showthread.php?p=9304431#post9304431

I believe this is long ago fixed in Ubuntu releases and doesn't need to be done manually.

A gold star for Gnome 3

When Canonical decided that Ubuntu would have a new user-interface, "Unity," to replace Gnome 2, there was a lot resistance and discontent in the Ubuntu user community.  Gnome 2 was a drop-down (or pop-up) menu system, harking back in a general way to the familiar Windows interface.

People who didn't like Unity could easily just go on using Gnome 2, at least for the foreseeable future (and switch to MATE after Gnome 2).

But the other alternative was to use Gnome 3, the official successor to Gnome 2.  Gnome 3 and Unity have quite a lot in common.  In fact, and it's clear that the Unity user interface is - broadly speaking - based on the same thinking as Gnome 3 about where user interfaces might be going.

As is documented in this blog, I was willing to give Unity a shot, and I even quite liked it.  But I had a lot of technical difficulties with Unity, so I decided to try Gnome 3.

Now I've been using Gnome 3 for nearly six months, and I'm very much at home with it.  And I'd like to put in a good word.  Recently, I had to work briefly with the old  Gnome 2 menus, and I couldn't believe how slow and fiddly the old interface now felt, after being thoroughly used to Gnome 3.

So, from me, at least, a gold star for Gnome 3.