Planet Plone

Afgelopen vrijdag vond er een lightning talks-sessie plaats bij ons op kantoor. In dit stukje blikken we heel kort terug en kondigen we de volgende editie alvast aan.

De sprekers zorgden weer voor een gevariëerd programma. De eerste onderwerpen waren:

• Sherpa; the next step in E-learning (Henk Bohlander)
• Subsidies in ICT (Rik van der Wel) (slides)
• Bitcoin (Rogier Eijkelhof / Patrick van der Meijde)
• LibreOffice (Cor Nouws)

Na een korte pauze ging het verder met:

• Sociaal projectmanagent (Ton Meeuwissen)
• GPX Energiebank (Egbert Bouwhuis)
• Waarom zijn we aangetrokken tot mooie dingen? (Patrick Kreling)

De sessie werd afgesloten met een borrel, waarbij er nog even werd nagepraat.

De volgende sessie zal plaatsvinden op vrijdag 26 september, weer vanaf 16:00. Opgeven, ook als spreker, kan via info@fourdigits.nl. Hopelijk zien we u dan.

(Foto's: Patrick Kreling)

Today I've released Morepath 0.3!

Morepath is a Python web framework that lets you write simple, flexible code for the modern web, and makes linking to things as easy as it should be.

New in Morepath 0.3 is the ability to write path directives that absorb the rest of the path. This is useful when you want to integrate a web server with a frontend that does its own client-side routing using the HTML 5 history API. In that case the server-side code needs an easy way to absorb all sub-paths so it can pass them to the frontend router.

Another feature is the ability to mark views as internal. This allows you to write a view that is not published on the web but is for reuse within other views.

Besides this Morepath 0.3 contains a number of bugfixes and documentation improvements. See CHANGES for more details.

If you like Morepath, please get in touch with me, either through the Morepath issue tracker or the #morepath IRC channel on Freenode. Let me know if there's interest in starting a mailing list too!

Sven Strack, Paul Roeland, Kurt Bendl and I -- with help from the assembled PSM masses -- got lots done on Plone 5 installers.

Decisions, Decisions

A major version change is a great opportunity to shake things up, so we spent the first half day of the conference detailing what we want to do for Plone 5. Paul, Sven and I have been tossing ideas at each other for months. With help from Kurt, it was decision time. In a head-spinning half day, we made all those decisions, then the fingers hit the keyboards.

Installers

Unified Installer

The next Unified Installer will try to be a bit kinder to new users by allowing them to choose configuration options interactively. You'll be able to use command-line options as before, but if you don't supply any command-line options, we'll start asking questions. When a console-dialog manager like whiptail or dialog is available, we'll use it to gather input. When not, we'll use text and read input from the console.

Steve wrote a wrapper that allows for menu, alert, confirmation, string input and password-input dialogs to be written with the same bash commands. whiptail ships with Ubuntu. dialog is available nearly everywhere else, including via homebrew on Macs.

The UI will also incorporate Sven's dependency checker and installer. Written with help from Giacomo Spettoli, Alexander Pisa and Paul, this tries to sniff out the Unix variant and make the right decisions on system packages. You can have it list out needed dependencies, then make the decision whether to install them yourself or have the program do it.

OS X Installer

Plone has had a GUI-based binary installer for OS X since Plone 2.5. It is, however, getting harder and harder to maintain as Apple continues to make OS X a less and less welcoming environment for GNU-tools-style development. This reached a new extreme when Mavericks' Python didn't work with Mavericks' c compiler without special environment flags that Apple is promising won't work soon.

We've decided to address this by thinking of OS X as a homebrew environment -- in the sense that Ubuntu is a debian-packages environment and CentOS/Redhat/etc. are RPM environments. We'll strip from the Unified Installer the code that's OS X-specific and instead require a system-packages kit available from homebrew. This allows us to socialize the problem of getting GNUish tools and libraries that actually work together.

For Mac users who aren't comfortable at the command line, Sven worked on a set of AppleScript script wrappers that will allow a double-clicky install that does everything from install Apple's current command-line tools to setting up the homebrew kit to downloading and running the Plone installer. (Anyone who suggests that we should have done this in Swift will be ruthlessly ridiculed.)

Windows Installer

Windows continues to be a problem for us. Giacomo has been heroically maintaining a high-quality Windows Installer, but he's been doing it for love of Plone rather than because he's a Windows fan. If we're to maintain a Windows installer, we need one or more people or companies that actually use Windows to step up to work on this installer. A great opportunity: Ross Patterson has a proof of concept working for a Plone WebPI (Microsoft Web Platform Installer) installer. But Ross also is not a real Windows user, and doesn't want to take on long-term ownership of the project.

Virtual Machine Images

We believe the importance of traditional installers will be fading with the rise of virtual machine / container deployment. We want to make it point-and-click easy to deploy Plone on AWS, DigitalOcean, OpenStack, Docker, VirtualBox or VMWare with Plone-community-blessed images. We anticipate making images available for testing/deployment  and for full-stack production configurations.

We decided on Packer as our fundamental tool for this. Sven's been testing Packer, and has proof-of-concept tests for automated builds of popular image formats. Note that behind the scenes, we'll be using the same provisioner for all the environments.

Server Provisioning

We had a fun Birds of a Feather breakout from the symposium to talk about repeatable server provisioning. Various folks in the group had experience with Puppet, Chef, Salt and Ansible. CalvinHP and Claytron, in particular, talked about managing lots of VMs with Salt/Minion and experiments with Ansible. Allegiances didn't change, but Paul, Kurt, Sven and I thought Ansible was the clear winner for the installer team's purposes. Deciding points for us were simplicity, use of ssh as a transport, and that it's written in Python.

The Installer Team plans on making Ansible playbooks and roles available in kits that are immediately useful for provisioning everything from testing/development to full-stack production servers. These playbooks will also provision the virtual images we create with Packer.

We hope we'll have lots of community involvement in creation and maintenance of Ansible roles for Plone-oriented deployment of apache/nginx, varnish, haproxy, supervisor and buildout.

The Ansible kits will be a good option for integrators that aren't satisfied with the choices made for the community VM images, but would like to benefit from known-working stack configuration. A typical workflow might be to fork the Plone Ansible kit, customize a configuration file, run the Ansible playbooks to set up a server, and turn on your Plone site. Those specialized integrators will be able to use the kit in conjunction with Packer to create their own VM images.

Skeleton Generators

We continued discussion and research that began in Brasília about a viable replacement for ZopeSkel/Templer, which are at best unevenly maintained and have dependencies on unmaintained code that's not keeping up with changes in Python's setuptools.

We considered a wider set of options, looking for a skeleton generator that's got larger community support, is actively maintained, and meets our basic needs. Claytron did heroic research/testing for us. At the end, only Mr.Bob remained standing. Mr.Bob the skeleton generator, not Mr.Bob the portable-toilet vendor.

Thanks to ...

UW Oshkosh was a great place to sprint, with easy movement from formal classrooms to dorm lounges, to restaurants and parks. Thanks to Kim Nguyen and the UWO crew for planning it all and taking care of us, and to the sprint sponsors: UW Oshkosh Office of the Provost, Six Feet Up, Wildcard Corp. and the Plone Foundation.

Thanks to Reinout van Rees at Nelen & Schuurmans in Utrecht for organizing the meetup.

for value in generator:
    yield value

from gevent import monkey; monkey.patch_all()

pip install elasticsearch

python -m unittest test

ls *py | entr -c python -m unittest test

from rest_toolkit import quick_serve, resource

from rest_toolkit.ext.sql import SQLResource

from whelk import shell, pipe
shell.git(...)
shell['2to3']...
shell.make('test', output_callback=...,
           run_callback=..., exit_callback=...
           raise_on_error=True)

git = shell.git
git.checkout('next')
if not git.diff('--quiet', 'master', raise_on_error=True):
    ...

pip install whelk

We needed to show a message to the end user when there were problems connecting to the LDAP server when authenticating, because people are muppets and don't remember their passwords or don't use a password manager.

Sometimes IT changes passwords because of policy/BOFH-ing and doesn't tell the user. Not to mention all the times IT take down a server for maintenance, or when someone needs a socket for the vacuum cleaner. Wow, someone's up for a surprise in the morning!

Now, what we've done is patch the authentication process and check if errors are logged. If so, display a message to the end user to bugger IT. Again.

Here's the gist.

patches.py:

from plone.api.portal import show_message
from cStringIO import StringIO
import logging
fake_log_file = StringIO()
aux_logger = logging.StreamHandler(fake_log_file)
aux_logger.setLevel(logging.ERROR)
ldap_logger = logging.getLogger('event.LDAPDelegate')
ldap_logger.addHandler(aux_logger)
msg = "Something went wrong while connecting to the LDAP server(s)"
def authenticateCredentials(self, credentials):
    fake_log_file.truncate(0)  # empty log to catch only current messages
    try:
        self._old_authenticateCredentials(credentials)
    except:
        raise
    finally:
        if fake_log_file.getvalue():
            # Assume something went wrong if it is logged.
            show_message(msg, self.REQUEST, type='error')
    return None, None

configure.zcml:

<monkey:patch
    description="Show message on LDAP errors."
    class="Products.LDAPMultiPlugins.LDAPPluginBase.LDAPPluginBase"
    original="authenticateCredentials"
    replacement=".patches.authenticateCredentials"
    preserveOriginal="true"
/>

If you have a traditional-style Plone theme that will continue to be in use for Plone 5 and beyond, I recommend updating it to use Diazo. This involves a few steps:

• Add a folder for your static content
• Move all custom CSS, JS, and images into the static folder
• Use Plone Default as the base theme
• Write an index.html and rules.xml
• Remove all skins overrides
• Reduce number of z3c.jbot overrides
• Move custom skins templates into view templates

One nice thing about these steps is that they don't all need to be done at the same time. They can be done gradually in pieces if you are short on time.

Add a folder for your static content

This folder is basically a browser resource, which plone.app.theming converts into a theme.

<plone:static 
    directory="static"
    type="theme"
    name="mytheme" />

Include a manifest.cfg in the directory for information about the theme that will display in the Theming control panel.

Move all custom CSS, JS, and images into the static folder

Move any CSS, JS, and images from your skins folders into the new static folder. They can be put into subdirectories to keep the file structure clean. The CSS and JS files can (and should) remain in the resource registries, the entries will need to be updated to include the new path, for example: ++theme++mytheme/main.css. If doing this from cssregistry.xml, remove the entries with the old path using remove=“true”. Paths to images from the CSS will need to be updated as well, and these can use relative paths as long as the applyPrefix is set to True in the CSS registry.

Use Plone Default as the base theme

Chances are, your theme is currently based on Sunburst. This should continue to work in Plone 5 initially, but eventually portal_skins will be removed completely (currently it is deprecated). In switching to Plone Default as the base, you can copy all styles from Sunburst and add them to your static folder. Entries in the CSS Registry will need to be updated by removing the files you copied, and re-adding them with the static path.

Write an index.html and rules.xml

These files will live directly inside the static folder. If you want to keep your theme looking the same, you can go to your site, View Source, copy all the code and put it in your index.html. Strip out any extra bits you don’t need. Then write your rules to map the dynamic content from Plone to this static html.

Remove all skins overrides, and reduce number of z3c.jbot overrides

portal_skins is being deprecated, and many of the skins templates are being moved into views, which means your customizations will no longer work in Plone 5. Now that you have a Diazo theme in place, see if your customizations can instead be done with a Diazo rule, so that you can remove the override completely. If you need to keep the override, use z3c.jbot instead of skins. When you upgrade to Plone 5, you will likely just need to change the name of the override to point to the original file’s new location. Try to do as much as you can with Diazo, to keep the number of overrides at a minimum. This will make upgrading easier.

Move custom skins templates into view templates

Let’s say you have a custom template for your homepage in skins. Move that template to the browser folder in your theme, and put this in the configure.zcml:

<browser:page
    for="*"
    name="homepage"
    template="homepageview.pt"
    permission="zope2.View" />

The template will now be accessible in the site at /@@homepage (path is based on the name you set). You will need to remove or update any code that assigns the template to the page. You can just specify the template as the href in a Diazo rule:

<replace
    css:theme="#content"
    css:content="#home"
    href="http://www.sixfeetup.com/blog//@@homepage"/>

There will still need to be adjustments once you upgrade to Plone 5. The steps in this outline are just my recommendations from my theming experience and investigation into Plone 5. I would love to get some opinions and suggestions from developers that have been actively working on Plone 5.

This post is long overdue since Plone 5 alpha 2 was released on 2014-04-20! It is also too short. I may do a followup blog entry to review more features, depending on how well this one is received. I also want to talk more about Plock.

When Plone 4 alpha 1 was released, I was so excited I immediately ran it in production. Now-a-days I use Pyramid for aclark.net but I still get excited about new Plone releases.

As such, I finally got around to testing the latest Plone 5 alpha with my deployment stack and the results were very impressive, so I thought I'd share.

$ pip install tmp virtualenv
$ cd `tmp`
$ virtualenv .
$ bin/pip install zc.buildout
$ bin/buildout init

[buildout]
extends = https://raw.github.com/plock/pins/master/plone-5-0

$ bin/buildout
$ bin/plone fg

Create Plone Site

The first thing you will notice is the improved "Create a Plone Site" page which now includes a timezone configuration option:

Barceloneta Theme

I selected the US/Eastern timezone and clicked Create Plone Site, shortly after which I was greeted with a beautiful new Plone 5 site in all its open source glory. Immediately you will notice a new theme and toolbar:

Plone Toolbar

Here is a quick tour of the new toolbar.

Folder contents

Edit button

Sharing settings

Workflow settings

Add content

Layout settings

User menu

Love it! Of all these, only the eye icon didn't function as expected. Not sure what it's supposed to do but I assume it's not done yet. Additionally, I wish the small Plone logo in the toolbar navigated "home" like the portal logo. Or maybe it will close the toolbar? Oh, the possibilities!

Plone Symposium Midwest 2014 training classes have been in session since last Friday, June 13, but we are now ready for our two main talk days!

Live Video Feed

If you are unable to attend PSM14 in person, please view our live video feed, on June 18 and 19, of our:

• keynotes
• talks
• lightning talks

On Twitter

Please follow us on Twitter

• @PloneSymp: https://twitter.com/PloneSymp
• at hashtag #psm14: https://twitter.com/hashtag/psm14?f=realtime

Netsight is working with Cosent and the Plone Intranet Consortium to design and build an open source social intranet platform.

At this relatively early stage of development, a small team from Netsight have been focussing on the architecture and initial implementation of some of the core components. With regular consultation amongst the Plone Intranet Consortium, we are hoping to establish a stable base intranet platform that can be extended and improved in future within the broader Plone Community.

One of the core requirements that came out of early discussions was the need for a flexible team/community workspace solution, that would allow teams of employees to communicate and collaborate effectively within their own area of the intranet. We established an architecture plan which distills the extensive permission framework within Plone into a set of distinct policies that control who can access a workspace, who can join a workspace, and what users can do once they are part of a workspace.

A few sprints in, and there are a number of key packages around this functionality that now have (or are nearing) an initial release:

experimental.securityindexing

An optimisation for catalog security re-indexing after role/workflow changes.

For more details see Matt Hamilton's separate blog post.

ploneintranet.workspace

• Provides workspace container and content workflow working in conjunction to provide flexible levels of content access
• Three levels of access controlled via a single ‘policies’ tab:
• External visibility (who can see the workspace)
• Join policy (who can join / add users to a workspace)
• Participation policy (default permissions given to members of a workspace)
• Built on dynamic PAS group/role plugins from collective.workspace 
• New ‘self-publisher’ role that allows users to publish their own content, but not the content of others (something that cannot be achieved with existing contributor/editor/reviewer roles). 
• Support for easily copying/moving members between workspaces

For more information see the ploneintranet.workspace architecture document.

ploneintranet.invitations

• Generic token framework to support dropbox/drive-style URL-based actions
• Demo implementation that allows user account login (and creation) via unique url

plone.api extension

• An update to api.content.transition that allows you to move an item to a target workflow state, regardless of the number of transitions required to get there.

For details see https://github.com/plone/plone.api/pull/164

Further reading:

• Designing an open source social intranet (Cosent Blog)
• experimental.securityindexing @ pypi
• ploneintranet.workspace documentation @ RTD
• ploneintranet.invitations documentation @ RTD