Charles Tapley Hoyt

The [International Society of Biocuration
(ISB)](https://www.biocuration.org/) partners with the journal
[Database](https://academic.oup.com/database) to get discounts for its
members when they publish there. This means the ISB's executive
committee needs to send a member list to the journal's editor.
Historically, this has been done manually by exporting the list from the
membership management plugin in the ISB Wordpress blog once per month
and emailing it to th This post is about my journey trying to automate
it

# 1. There must be an API for this

Wordpress has a [programmatic
API](https://developer.wordpress.org/rest-api/), and specifically an
endpoint to list
[users](https://developer.wordpress.org/rest-api/reference/users/).

After logging into the ISB's Wordpress site, I was able to list users by
navigating to the endpoint in browser
https://biocuration.org/wp-json/wp/v2/users. Note: this page won't work
for you unless you're on the EC and have admin powers. I wanted to
replicate accessing this page through a Python script, so I was
suggested by the [official Wordpress
documentation](https://developer.wordpress.org/rest-api/using-the-rest-api/authentication/)
to create an application password and then use simple HTTP
authentication. The docs said try this, after replacing the username
with the Wordpress account (for me, that's `cthoyt`{.language-plaintext
.highlighter-rouge}) and the application password, which is a string
with the form of `XXXX XXXX XXXX XXXX XXXX XXXX`{.language-plaintext
.highlighter-rouge}. The documentation suggested doing the following
curl command, which works:

:::: {.language-shell .highlighter-rouge}
::: {.highlight}
``` highlight
curl --user "USERNAME:PASSWORD" https://biocuration.org/wp-json/wp/v2/users
```
:::
::::

If you want to translate this to Python, it looks like this.

:::: {.language-python .highlighter-rouge}
::: {.highlight}
``` highlight
import pystow, base64, requests

username = pystow.get_config("isb", "wordpress_username", raise_on_missing=True)
password = pystow.get_config("isb", "wordpress_password", raise_on_missing=True)

token = base64.b64encode(f"{username}:{password}".encode()).decode("utf-8")

res = requests.get(
    'https://www.biocuration.org/wp-json/wp/v2/users',
    headers={"Authorization": f"Basic {token}", "user-agent": "curl"},
)
```
:::
::::

Luckily, encoding the username/password string with basic authorization
is such a common pattern that the `request`{.language-plaintext
.highlighter-rouge} (and any other modern library) has a nice par for
this.

:::: {.language-python .highlighter-rouge}
::: {.highlight}
``` highlight
import pystow, requests

username = pystow.get_config("isb", "wordpress_username", raise_on_missing=True)
password = pystow.get_config("isb", "wordpress_password", raise_on_missing=True)

res = requests.get(
    'https://www.biocuration.org/wp-json/wp/v2/users',
    auth=(username, password),
    headers={"user-agent": "curl"}
)
```
:::
::::

I burned a ton of time with this because it turns out that Wordpress
blocks both the user agents for the `request`{.language-plaintext
.highlighter-rouge} and `httpx`{.language-plaintext .highlighter-rouge}
libraries. Therefore, you have to explicitly set the
`user-agent`{.language-plaintext .highlighter-rouge} header to something
else, or you get a HTTP 403 forbidden error.

# 2. The API doesn't do what I wanted

It turns out that by default, all the users that never made a post get
filtered out. This is bad since the only users who are making posts on
the ISB are the handful who have or are currently on the executive
committee.

So, obviously, the next step is to start injecting in PHP code to change
how the API works. I found a
[comment](https://github.com/WP-API/WP-API/issues/2300#issuecomment-299202391)
from Tim Jensen, a Wordpress developer, that suggests adding the
following hooks into the Wordpress theme's
`functions.php`{.language-plaintext .highlighter-rouge} file by
navigating to
https://www.biocuration.org/wp-admin/theme-editor.php?file=functions.php&theme=executive:

:::: {.language-php .highlighter-rouge}
::: {.highlight}
``` highlight
function remove_has_published_posts_from_api_user_query($prepared_args, $request)
{
    unset($prepared_args['has_published_posts']);
    return $prepared_args;
}
add_filter('rest_user_query', 'remove_has_published_posts_from_api_user_query', 10, 2);
```
:::
::::

This actually worked! But, it wasn't the end of the story.

# 3. Full names are part of the plugin's metadata

It turns out that the user data model isn't all that comprehensive in
Wordpress. What we really needed was the full name and email address for
each person, and that was stored in an auxillary SQL table created and
managed by the memberships plugin. Note that the plugin lives on top of
Wordpress's first-party users list, and doesn't modify the API access to
the user list.

So obviously, the next step was to add new API endpoints. I learned PHP
at the ripe age of 14 and when I learned some other languages, I never
looked back, so this wasn't so easy. Plus, Wordpress has a huge set of
functions and idioms on top. With a little help from ChatGPT, I was able
to write a new API endpoint to add to
`functions.php`{.language-plaintext .highlighter-rouge} that got users
whose membership expiration date is in the future and joined in the
metadata for their first and last names:

:::: {.language-php .highlighter-rouge}
::: {.highlight}
``` highlight
function export_pmp_members_to_csv() {
    global $wpdb;

    // Set headers to make the response downloadable as a CSV file
    header('Content-Type: text/csv');
    header('Content-Disposition: attachment; filename="members_export.csv";');
    header('Pragma: no-cache');
    header('Expires: 0');

    // Open output stream (in-memory file)
    $output = fopen('php://output', 'w');

    if (!$output) {
        return new WP_Error('file_error', 'Unable to generate CSV file.');
    }

    // Add CSV header row
    fputcsv($output, ['id', 'first', 'last', 'email', 'level', 'start', 'ends']);

    // Query to get PMP members and their membership details
    $results = $wpdb->get_results("
        SELECT 
            u.ID, 
            um_first_name.meta_value AS first_name, 
            um_last_name.meta_value AS last_name, 
            u.user_email, ml.name as membership_name, 
            mu.startdate, 
            mu.enddate
        FROM {$wpdb->users} u
        LEFT JOIN {$wpdb->prefix}pmpro_memberships_users mu ON u.ID = mu.user_id
        LEFT JOIN {$wpdb->prefix}pmpro_membership_levels ml ON mu.membership_id = ml.id
        LEFT JOIN {$wpdb->prefix}usermeta um_first_name ON u.ID = um_first_name.user_id 
            AND um_first_name.meta_key = 'first_name'
        LEFT JOIN {$wpdb->prefix}usermeta um_last_name ON u.ID = um_last_name.user_id 
            AND um_last_name.meta_key = 'last_name'
        WHERE mu.membership_id IS NOT NULL
            AND (mu.enddate > CURDATE() OR mu.enddate IS NULL)
    ");

    // Loop through results and write each row to the CSV
    foreach ($results as $row) {
        fputcsv($output, [
            $row->ID, 
            $row->first_name,
            $row->last_name,
            $row->user_email, 
            $row->membership_name, 
            $row->startdate, 
            $row->enddate
        ]);
    }

    // Close the output stream
    fclose($output);

    // Terminate script to ensure no extra output is sent
    exit;
}

function register_pmp_export_route() {
    // the actual path to the endpoint isn't important here
    register_rest_route('path/to/endpoint/', '/export-members', [
        'methods' => 'GET',
        'callback' => 'export_pmp_members_to_csv'
        'permission_callback' => function() {
            // Optionally, restrict access to logged-in users with specific capabilities
            return current_user_can('manage_options'); // Restrict to admin users
        }
    ]);
}
add_action('rest_api_init', 'register_pmp_export_route');
```
:::
::::

# 4. Automate it

Why stop at just being able to export the sheet? I wanted to go another
mile and make sure that Wordpress sends an email to the right person at
the journal on a monthly basis.

:::: {.language-php .highlighter-rouge}
::: {.highlight}
``` highlight
/ Function to generate and send the CSV via email
function send_pmp_members_csv_via_email() {
    global $wpdb;

    // Create a temporary file to store the CSV
    $tmp_file = tempnam(sys_get_temp_dir(), 'pmp_csv_') . '.csv';
    $output = fopen($tmp_file, 'w');

    if (!$output) {
        return; // Handle error appropriately
    }

    // Add CSV header row
    fputcsv($output, ['ID', 'Username', 'First Name', 'Last Name', 'Email', 'Membership Level Name', 'Joined Date', 'Expires Date']);

    // Query to get PMP members and their membership details
    $results = $wpdb->get_results("
        SELECT u.ID, 
            u.user_login, 
            um_first_name.meta_value AS first_name, 
            um_last_name.meta_value AS last_name,
            u.user_email, 
            ml.name as membership_name
            mu.startdate,
            mu.enddate,
        FROM {$wpdb->users} u
        LEFT JOIN {$wpdb->prefix}pmpro_memberships_users mu ON u.ID = mu.user_id
        LEFT JOIN {$wpdb->prefix}pmpro_membership_levels ml ON mu.membership_id = ml.id
        LEFT JOIN {$wpdb->prefix}usermeta um_first_name ON u.ID = um_first_name.user_id 
            AND um_first_name.meta_key = 'first_name'
        LEFT JOIN {$wpdb->prefix}usermeta um_last_name ON u.ID = um_last_name.user_id 
            AND um_last_name.meta_key = 'last_name'
        WHERE mu.membership_id IS NOT NULL
        AND (mu.enddate > CURDATE() OR mu.enddate IS NULL)
    ");

    // Write data to CSV
    foreach ($results as $row) {
        fputcsv($output, [
            $row->ID, 
            $row->user_login, 
            $row->first_name,
            $row->last_name,
            $row->user_email, 
            $row->membership_name, 
            $row->startdate, 
            $row->enddate
        ]);
    }

    // Close the file
    fclose($output);

    // Prepare email
    $to = ''; // Replace with recipient email
    $subject = 'ISB Monthly Members Report';
    $message = 'Please find the attached CSV file containing the list of active members of the International Society of Biocuration as of today.';
    $headers = array('Content-Type: text/csv; charset=UTF-8');

    // Send email with attachment
    wp_mail($to, $subject, $message, $headers, $tmp_file);

    // Clean up temporary file
    unlink($tmp_file);
}

// Schedule the cron job to run monthly
function schedule_monthly_csv_email() {
    if (!wp_next_scheduled('monthly_pmp_csv_email')) {
        wp_schedule_event(time(), 'monthly', 'monthly_pmp_csv_email');
    }
}
add_action('wp', 'schedule_monthly_csv_email');

// Hook the function to the cron event
add_action('monthly_pmp_csv_email', 'send_pmp_members_csv_via_email');
```
:::
::::

Hopefully I don't have to write anymore PHP for a long time :)


The International Society of Biocuration (ISB) partners with the journal Database to get discounts for its members when they publish there. This means the ISB’s executive committee needs to send a member list to the journal’s editor.

Programmatic Access to a Wordpress User List

Ingo Pfennigstorf

I was quite sceptical if an online conference could make me pay
attention in the same way a "physical" conference could. Spoiler: It
could.

There were some minor issues, such as not so perfect sound, but it was
sufficient.

The conference took place at an online platform (hopin). All
[sessions](https://live.symfony.com/2021-world/) were pre-recorded and
made available for all attendees after the conference. During the two
conference days, the talks started again after seven hours to also give
developers from all timezones the opportunity to attend all talks.

Each talk has been subtitled in english, that helped me a lot to follow
some speakers.

I will focus on my highlights of that conference in this article.

## Keynote

The Keynote has been held by Symfony-inventor [Fabien
Potencier](http://fabien.potencier.org/) and for quite some surprise
announced, that Symfony is going to drop the Long-Term-Support (LTS)
releases. So, in order to stay up-to-date Symfony applications should be
updated every 6 month, when a new version is released.

![\#NoLTS](../assets/images/sfcon2021/symfony-lts.webp)

Fabien also announced that PostgreSQL is now the preferred way to store
data in Symfony. That also applies for messages and locks.

![PostgreSQL](../assets/images/sfcon2021/pgsql.webp)

## Symfony Forms: Advanced Use Cases

The talk by [Alexandre Salomé](https://alexandre.salome.fr/) was not so
much about advanced use cases, but an interesting and nice walk-through
the Symfony
[forms](https://symfony.com/doc/current/forms.html)-component with a lot
of gems.

![Forms](../assets/images/sfcon2021/forms.webp)

I immediately recognized some simplifications for things we implemented
in projects and had the feeling to now have a better understanding of
forms, where I lacked some knowledge.

## Symfony Notifier Demystified

[Jan Schädlich](http://www.janschaedlich.de/) introduced the symfony
[notifier](https://symfony.com/doc/current/notifier.html) component and
its versatility and simplicity. The component can be used to write
Notifications to different channels, such as flash messages on the
website, console or even - depending on the configuration - via SMS,
email, or to chat channels.

## Keynote - You're not in IT

[Stefan Koopmanschap](https://twitter.com/skoop) led a talk, about the
misunderstanding that we are working in the IT business but more in a
learning business. He took us on a journey through his career and
achievements in being a developer and open source maintainer.

![skoop](../assets/images/sfcon2021/skoop.webp)

Important for me were all the achievements he 'unlocked' and that for
being a successful developer there may be more to life than coding, but
also helping others by spreading the word or being a supporter.

Great Talk!

## Keynote - Towards Digital Sustainability

A quite unexpected (at least for me) keynote about digital
sustainability and green IT by [François
Zaninotto](https://twitter.com/francoisz) gave important impulses to
think about when developing applications.

His company developed a [service](https://greenframe.io/) for measuring
the carbon footprint of websites.

He once ran a widely adopted library, called
[faker](https://github.com/fzaninotto/Faker) on that he [dropped the
maintenance](https://marmelab.com/blog/2020/10/21/sunsetting-faker.html)
for ecological reasons.

## Cypress, the E2E must learn from the past

[Guillaume Loulier](https://www.guillaumeloulier.fr/en/) introduced
[cypress](https://www.cypress.io/) as end-to-end testing framework and
its integration into Symfony.

It has a better integration than Panther and also supports more
features. The talk was great for people, that want to dive into the
world of e2e-testing.

## Runtime component: The game changer

The talk by [Tobias Nyholm](https://tnyholm.se/) was one of my favourite
talk at this conference. The sound and video were great, one could
immediately see that Tobias is an experienced Speaker.

![Nyholm Setup](../assets/images/sfcon2021/nyholm.webp)

He explained the newly integrated symfony/runtime component, whose
benefits I didn't understand upon reading the
[article](https://symfony.com/blog/new-in-symfony-5-3-runtime-component)
about it. The runtime component - in short - solves problems where
different PHP runtimes are used for serving content, such as
[RoadRunner](https://roadrunner.dev/),
[Swoole](https://www.swoole.co.uk/) or [bref](https://bref.sh/) by
decoupling the application from the global state.

## Closing Keynote - A Dynamic Frontend with Twig & Zero JavaScript? Say Hello to...

The closing keynote was presented by [Ryan
Weaver](https://twitter.com/weaverryan). Ryan introduced a new way of
reusing Twig components for a symfony application. The
[component](https://github.com/symfony/ux-twig-component) is still in an
early stage, but looks very pro

![Twig Components](../assets/images/sfcon2021/component.webp)

He showed a lot of code, and the approach and code examples looked great
and were comprehensible. I'm really looking forward for using them, as I
needed such things often in the past and mostly used a
`RenderController`{.language-plaintext .highlighter-rouge} approach.


I was quite sceptical if an online conference could make me pay attention in the same way a “physical” conference could. Spoiler: It could. There were some minor issues, such as not so perfect sound, but it was sufficient. The conference took place at an online platform (hopin). All sessions were pre-recorded and made available for all attendees after the conference.

SymfonyWorld Online 2021 Summer Edition

Stephen Royle

If I struggle to find the answer to something, it must be worth a post
to help out others. So, here's where to find the `php.ini` file on a
standard WordPress install on Raspberry Pi.

I installed a LAMP stack and WordPress on a Raspberry Pi following these
excellent
[instructions](https://raspberrytips.com/wordpress-on-raspberry-pi/).
The aim was to import our lab's ELN onto a fresh WordPress install. To
do this, go to the dashboard of your old WordPress and select Tools \>
Export from the sidebar and select All. This will download an XML file
that can be used to import posts, comments, media etc. into another
WordPress installation. After setting up my new WordPress on the Pi, I
chose Tools \> Import... and...

## The problem {#the-problem .wp-block-heading}

On a fresh install, the limit for the XML import is 2MB. So, a large XML
file will not import. The following error pops up.

::: {.wp-block-syntaxhighlighter-code}
``` {.brush: .plain; .title: .; .notranslate title=""}
File is empty. Please upload something more substantial. This error could also be caused by uploads being disabled in your php.ini or by post_max_size being defined as smaller than upload_max_filesize in php.ini.
```
:::

It's just a matter of editing the `php.ini` file. **But where is it?**

## The answer {#the-answer .wp-block-heading}

It's in `/etc/php/7.3/apache2/php.ini`

To edit it do

::: {.wp-block-syntaxhighlighter-code}
``` {.brush: .bash; .title: .; .notranslate title=""}
sudo nano /etc/php/7.3/apache2/php.ini
```
:::

and then ctrl+W to find the parameters that need changing, I went for:

::: {.wp-block-syntaxhighlighter-code}
``` {.brush: .plain; .title: .; .notranslate title=""}
upload_max_filesize = 128M
post_max_size = 128M
```
:::

ctrl+X and Y to save. Then you need to restart apache2 with
`sudo service apache2 restart` or perform a reboot.

## The wrong answers {#the-wrong-answers .wp-block-heading}

1.  There is no `php.ini` file created by a WordPress installationand
    there are plenty of posts out there which tell you to create one in
    `wp-admin` or `wp-content` or even in root folder
    (`/var/www/html/`). This did not work for me.
2.  Locate `php.ini` on the command line. Over ssh, you can do
    `php -i | grep ini` and this will tell you which `php.ini` file is
    being used *on the command line*. I got `/etc/php/7.3/cli/php.ini`
    but editing this did not work.
3.  Many answers on the web refer to other types of installations. My
    answer above works for WordPress on a Raspberry Pi LAMP stack. If
    you are running windows or at a webhost, the answer maybe different.

How did I find the php.ini file? I made a file called `temp.php` in
`/var/www/html/` that said

::: {.wp-block-syntaxhighlighter-code}
``` {.brush: .php; .title: .; .notranslate title=""}
<?php phpinfo(); ?>
```
:::

And then in a browser, I went to `http://myipaddress/temp.php` where
myipaddress is the ip of my Raspberry Pi that runs WordPress. The
browser shows a table with lots of info. Look for **Loaded Configuration
File** and you will see `/etc/php/7.3/apache2/php.ini` further down you
will see the attributes that you need to change. As you update the file
and restart the service, you can refresh this page to check your
settings have taken hold.

---

This post is part of a [series of
tips](https://quantixed.org/tag/tftb/).


If I struggle to find the answer to something, it must be worth a post to help out others. So, here’s where to find the php.ini file on a standard WordPress install on Raspberry Pi.  I installed a LAMP stack and WordPress on a Raspberry Pi following these excellent instructions. The aim was to import our lab’s ELN onto a fresh WordPress install.

Tips from the Blog XV: locating the php.ini file for WordPress on Raspberry Pi

University of Glasgow

Roderic Page

![mendeley.png](http://lh3.ggpht.com/_Gct8lVAxKqQ/TGOqbgGhEbI/AAAAAAAAAtM/c63RwupveNg/mendeley.png?imgmax=800){border="0"
width="128" height="128" align="right"}Following on from my [earlier
post](http://iphylo.blogspot.com/2010/08/more-on-mendeley-api.html)
about the Mendeley API, I\'ve bundled up my code for OAuth access to the
Mendeley API for anyone who\'s interested in playing with the API using
PHP. You can [browse the
code](http://code.google.com/p/iphylo/source/browse/#svn/trunk/mendeley-oapi-php)
on Google Code, or grab a [tarball
here](http://code.google.com/p/iphylo/downloads/detail?name=mendeley-oapi-php.tgz).
You\'ll need a consumer key and a consumer secret from Mendeley for the
demos to work, and if you\'re behind a HTTP proxy you\'ll have to tweak
the code (this is explained in the ReadMe.txt file that comes with the
code).  
  
The code is pretty rough, and doesn\'t use all the Mendeley API calls,
but I\'ve other things to do, and it felt like a case of either bundle
this up now, or it will get lost among a host of other projects. The
Mendeley API still feels woefully under-developed. I\'d be more
interested in developing this client further if the API was powerful
enough to do the kinds of things I\'d like to do.


Following on from my earlier post about the Mendeley API, I've bundled up my code for OAuth access to the Mendeley API for anyone who's interested in playing with the API using PHP. You can browse the code on Google Code, or grab a tarball here.

Mendeley API PHP client

A little while ago I came across [Wikisource](http://wikisource.org/),
and it dawned on me that this is a model for
[BHL](http://www.biodiversitylibrary.org). To quote from the Wikisource
web site:  

> Wikisource is an online library of free content publications,
> collected and maintained by our community. We now have 140,596 texts
> in the English language library. See our inclusion policy and help
> pages for information on getting started, and the community portal for
> ways you can contribute. Feel free to ask questions on the community
> discussion page, and to experiment in the sandbox.

  
Much of their content comes from the Internet Archive (as does BHL\'s),
and Wikisource have developed extensions for Mediaiwki to do some cool
things, such as extract text and images from DjVu files. If you haven\'t
come across [DjVu](http://en.wikipedia.org/wiki/DjVu) before, it\'s a
format designed to store scanned documents, and comes with some powerful
open source tools for extracting images and OCR text. Wikisource can
take a DjVu file, extract images, thumbnails and text, creating
side-by-side displays where users can [edit and correct OCR
text](http://en.wikisource.org/wiki/Page:BulldogDrummondSapper.djvu/12):  
  

::: {style="text-align:center;"}
![wikisource.png](http://lh6.ggpht.com/_Gct8lVAxKqQ/S45GCfgOWQI/AAAAAAAAAnY/cFtAxdRgI98/wikisource.png?imgmax=800){border="0"
width="425"}
:::

  
  
So, like a fool, I decided to try and install some of these tools
locally and see if I could do the same for some BHL content. That\'s
when the \"fun\" started. Normally Mediawiki is pretty easy to set up.
There are a few wrinkles because my servers live behind an institutional
HTTP proxy, so I often need to tweak some code (such as the
[OpenID](http://www.mediawiki.org/wiki/Extension:OpenID) extension,
which also needs [a fix for PHP
5.3](http://sourcecookbook.com/en/recipes/60/janrain-s-php-openid-library-fixed-for-php-5-3-and-how-i-did-it)),
but installing the extensions that underlie Wikisource wasn\'t quite so
straightforward.  
  
**DjVu**  
![djvu.png](http://lh5.ggpht.com/_Gct8lVAxKqQ/S442qWA5C6I/AAAAAAAAAnQ/Y-8ziIx37eg/djvu.png?imgmax=800){border="0"
width="128" height="128" align="right"}  
The first step is supporting DjVu files in Mediawiki. This seems
straightforward (see [How to use DjVu with
MediaWiki](http://www.mediawiki.org/wiki/Manual:How_to_use_DjVu_with_MediaWiki)).
First off you need the DjVu tools. I use Mac OS X, so I get these
automatically if I install
[DjView](http://djvu.sourceforge.net/djview4.html). The tools reside in
`Applications/DjView.app/Contents/bin` (you can see this folder if you
Control+click on the DjView icon and choose \"Show Package Contents\"),
so adding this path to the name of each tool DjVu tool Mediaiwiki needs
takes care of that.  
  
But I also need [NetPbm](http://netpbm.sourceforge.net/), and now the
pain starts. NetPbm won\'t build on Mac OS X, at least not out of the
box on Snow Leopard. It makes assumptions about Unix that Mac OS X
doesn\'t satisfy. After some compiler error messages concerning missing
variables that I eventually traced to `signal.h` I gave up and installed
[MacPorts](http://www.macports.org/), which has a working version of
NetPbm. MacPorts installed fine, but it\'s a pain having multiple copies
of the same tools, one in `/usr/local`, and one in `/opt/local`.  
  
OK, now we can display DjVu files in Mediawiki. It\'s small victories
like this which leads to over confidence\...  
  
**Proofread Page**  
Next comes the [Proofread
Page](http://www.mediawiki.org/wiki/Extension:Proofread_Page) extension,
which provides the editing functionality. This seemed fairly
straightforward, although the documentation referred to a SQL file
(`ProofreadPage.sql`) that doesn\'t seem to exist. More worringly, the
documentation also says:  

> If you want to install it on your own wiki, you will need to install a
> 404 handler for generating thumbnails, such as WebStore.

  
This seems fine, except the page for
[WebStore](http://www.mediawiki.org/wiki/Extension:WebStore) states:  

> The WebStore extension is needed by the ProofreadPage extension.
> Unfortunately, documentation seems to be missing completely. Please
> add anything you know about this extension here.

  
Then there are the numerous statements \"doesn\'t work\" scattered
through the page. So, I installed the extension and hoped for the best.
It didn\'t work. As in, really, really didn\'t work. It took an hour or
so of potentially fatal levels of blood pressure-inducing frustration to
get to the bottom of this.  
  
**WebStore**  
Now, Webstore is a clever idea. Basically, the Proofread Page extension
will need thumbnails of images in potentially varying sizes, and creates
a link to the image it wants. Since that image doesn\'t exist on the web
site the web server returns 404 Not Found, which normally results in a
[page like this](http://biostor.org/error.html). Instead, we tell the
web server (Apache) that WebStore will handle 404\'s. If the request is
for an image, Webstore creates the image file, streams it to the web
browser, then deletes the file from disk. Essentially WebStore creates a
web server for images
([Webdot](http://iphylo.blogspot.com/2006/04/render-dot-files-on-fly-on-mac-os-x.html)
uses much the same trick, but without the 404 handler). Debugging a web
server called by another web server is tricky (at least for a clumsy
programmer like me), but by hacking the Webstore code (and switching on
Mediawiki debug logging) I managed to figure out that Webstore seemed to
be fetching and streaming the images fine, but they simply didn\'t
appear in the wiki page (I got the little broken image icon instead). I
tried alternative ways of dumping the image file to output, adding HTTP
headers, all manner of things. Eventually (by accident, no idea how it
happened) I managed to get an image URL to display in the Chrome web
browser, but it wasn\'t an image(!) \-- instead I got a PHP warning
about two methods in the class `DjVuHandler` (`mustRender` and
`isMultiPage`) not being consistent with the class they inherit from.
WTF?! Eventually I found the relevant file (DjVu.php in `includes/media`
in the Mediawiki folder), added the parameter `$file` to both methods,
and suddenly everything works. At this point I didn\'t know whether to
laugh or cry.  
  
**OCR text**  
There are some issues with OCR text from Internet Archive DjVu files.
There are some extraneous characters (new lines, etc.) that I need to
filter, and I\'ll probably have to deal with hyphenation. It looks
fairly straightforward to edit the proofing extension code to handle
these situations.  
  
**Semantic Mediawiki**  
Having got the proofing extension working, I then wanted to install the
Semantic Mediawiki extensions so that I could support basic inference on
the wiki. I approached this with some trepidation as there are issues
with Mediawiki namespaces, but everything played nice and so far things
seem to be working. Now I can explore whether I can combine the proofing
tools from Wikisource with the code I\'ve developed for
[iTaxon](http://iphylo.blogspot.com/search?q=itaxon).  
  
**BioStor**  
So, having got something working, the plan is to integrate this with
[BioStor](http://biostor.org/). One model I like is the video site
[Metacafe](http://www.metacafe.com/). For each video Metacafe has a
custom web page(e.g.,
[http://www.metacafe.com/watch/4137093](http://www.metacafe.com/watch/4137093/mr_bean_hitch_hiking_to_the_golf_course/))
with an [Edit Video
Details](http://wikicafe.metacafe.com/f/en/index.php?action=formedit&title=4137093#tab=Video_type)
link that takes you to a Semantic Mediawiki page where you can edit
metadata for the video. I envisage doing something similar for BioStor,
where my existing code provides a simple view of an article (perhaps
with some nice visualisations), with a link to the corresponding wiki
page where you can edit the metadata, and correct the OCR text.  
  
**Lessons**  
In the end I got there, although it was a struggle. Mediawiki is a huge,
complicated bit of software, and is also part of a larger ecosystem of
extensions, so it has enormous power. But there are lots of times when I
think it would be easier if I wrote something to replicate the bit of
functionality that I want. For example, side-by-side display of text and
images would be straightforward to do. But once you start to think about
supporting mark-up, user authentication, recording edit history, etc.,
the idea of using tools others have developed becomes more attractive.
And the code is open source, which means if it doesn\'t work there\'s a
fighting chance I can figure out why, and maybe fix it. It often feels
harder than it should be, but I\'ll find out in the next few days
whether yesterday\'s exertions were worth it.  
  


A little while ago I came across Wikisource, and it dawned on me that this is a model for BHL. To quote from the Wikisource web site:  Much of their content comes from the Internet Archive (as does BHL's), and Wikisource have developed extensions for Mediaiwki to do some cool things, such as extract text and images from DjVu files.

Setting up a local Wikisource

Thinking about ways to improve the performance of some of my web servers
I\'ve begun to toy with Memcached. These notes are to remind me how to
set it up (I\'m using Mac OS X 10.5, Apache 2 and PHP 5.2.10, as
provided by Apple). Erik\'s blog post [Memcached with PHP on Mac OS
X](http://af-design.com/blog/2008/10/18/memcached-with-php-on-mac-os-x/)
has a step-by-step guide, based on the post [Setup a Memcached-Enabled
MAMP Sandbox
Environment](http://www.lullabot.com/articles/setup-memcached-mamp-sandbox-environment)
by Nate Haug, and I\'ve basically followed the steps they outline.  

1.  **Install the Memcached service on Mac OS X:** Follow the
    instructions in Nate Haug\'s
    [post](http://www.lullabot.com/articles/setup-memcached-mamp-sandbox-environment).

2.  **Install Memcache PHP Extension:** Apple\'s PHP doesn\'t come with
    the PECL package for memcache so [download
    it](http://pecl.php.net/package/memcache). To compile it go:  

        phpize
        ./configure
        make
        sudo make install

      
    One important point. If you are running 64-bit Mac OS X (as I am),
    ./configure by itself won\'t build a usable extension. However, [a
    comment by
    Matt](http://af-design.com/blog/2008/10/18/memcached-with-php-on-mac-os-x/#comment-719)
    on Erik\'s original post provides the solution. Instead of just
    ./configure, type this at the command prompt:  

    ::: {style="font-family:Courier;font-size:12px;"}
      
    MACOSX_DEPLOYMENT_TARGET=10.5 CFLAGS=\"-arch ppc -arch ppc64 -arch
    i386 -arch x86_64 -g -Os -pipe -no-cpp-precomp\" CCFLAGS=\"-arch ppc
    -arch ppc64 -arch i386 -arch x86_64 -g -Os -pipe\" CXXFLAGS=\"-arch
    ppc -arch ppc64 -arch i386 -arch x86_64 -g -Os -pipe\"
    LDFLAGS=\"-arch ppc -arch ppc64 -arch i386 -arch x86_64
    -bind_at_load\" ./configure
    :::

      
      
    Then follow the rest of Erik\'s instructions for adding the
    extension to your copy of PHP.

3.  **Restart Apache:** You can do this by restarting Web sharing in
    System preferences. Use the
    [phpinfo();]{style="font-family:Courier;"} command to check that the
    extension is working. You should see something like this:  

    ::: {style="text-align:center;"}
    ![memcache.png](http://lh3.ggpht.com/_Gct8lVAxKqQ/Stm5R4EuiFI/AAAAAAAAAkc/jlbYMibXRYQ/memcache.png?imgmax=800){border="0"
    width="380"}
    :::

      
    If you don\'t see this, something\'s gone wrong. The Apache web log
    may help (for example, that\'s where I discovered that I had the
    problem reported by several people who commented on [Erik\'s
    post](http://af-design.com/blog/2008/10/18/memcached-with-php-on-mac-os-x/).

4.  You can start the memcached daemon like this:  

        # /bin/sh
        memcached -m 1 -l 127.0.0.1 -p 11211 -d

  
Now, I just need to explore how to actually use this\...


Thinking about ways to improve the performance of some of my web servers I've begun to toy with Memcached. These notes are to remind me how to set it up (I'm using Mac OS X 10.5, Apache 2 and PHP 5.2.10, as provided by Apple). Erik's blog post Memcached with PHP on Mac OS X has a step-by-step guide, based on the post Setup a Memcached-Enabled MAMP Sandbox Environment by Nate Haug, and I've basically followed the steps they outline.

Messaggi di Rogue Scholar

Programmatic Access to a Wordpress User List

SymfonyWorld Online 2021 Summer Edition

Tips from the Blog XV: locating the php.ini file for WordPress on Raspberry Pi

Mendeley API PHP client

Setting up a local Wikisource

Memcached, Mac OS X, and PHP