Planet Drupal

The author's copies of Pro Drupal 7 Development, Third Edition arrived today.

Although this book is thinner physically than the second edition, the pages are actually wider and there are more of them: 689 pages all told vs. 667 in the second edition. That's the third edition on the bottom in the picture below.

Here's the traditional picture of me and the little guy. Time flies!

Hear Ye! Hear Ye!

You are all invited to the Drupal Association's second Town Hall meeting!

read more

In the past years I’ve had discussion about a Drupal academic initiative with many people. 2011 seems to be an excited moment to see the many discussion turn into actions. A good project needs to have at least a small user group to help shape the initiative. To do so we plan three events during different Drupal events. The first one is very "ad hoc" and will happen soon: Devdays 5 and 6 February. You can join the discussion on the session pageThe second meeting (BoF) is planned during Drupalcon Chicago in March. If all goes well this will result in a research-track during Drupalcon London in August. There is still much to do and any help is welcome.

Tweet

I tend to spend a lot of time tracking down and fixing bugs on our websites. After doing this for a while, I developed some tricks to quickly diagnose the problem. I'm going to walk you through a recent bug report and fix that I had to do and how I fixed it. Unfortunately most bug fixing involves reading code though so if you aren't a programmer, you might get a little lost.

Last Thursday we had a bug report come in that the coupons on the website weren't working and that it appeared all the coupons had disappeared from the site.

The first step to fixing any bug is figuring out a way to reliably replicate it. If you can't replicate it, you can't fix it. This bug had a very obvious way it was being manifested and that was on the Coupons page, nothing was showing up. To verify the bug, I made a copy of our live environment and opened up the database and took a peek at the uc_coupons table. There were 8 entries in the table so the Coupons admin page should have showed 8 entries. Now I have to figure out why the coupons aren't showing up.

The next thing is to check and make sure this is the latest version of the module so I check the project page to see if there are any updates. I'm one minor point behind but I update anyways just in case this was causing the problem. It doesn't fix the problem.

I also checked the issue queue for any similar bugs and nothing comes up. Often times I find the bug already reported and a patch to try out to see if it will fix the problem. This time though there were no bugs even close. While some might be tempted to report the bug right away, I'd like to dig in a little deeper and see if this isn't a result of something we've done since we have a fair amount of custom modules and a fairly complex site.

My next step usually involves finding the code that is being run for a page. One trick I like to use is the menu_router table. When I go to the admin page where the coupons show up, I see the url is "admin/store/coupons." By opening up the menu_router table and finding the row for the path "admin/store/coupons," I can see what function is the page callback. In this case I can see that the function is uc_coupon_display() in the file "sites/all/modules/contrib/uc_coupon/uc_coupon.admin.inc"

Now I need to see what is actually going on during the callback so I open up my text editor and open the uc_coupon.admin.inc file and look for the uc_coupon_display() function. Right near the top of the function is a pager_query that should return a result.

  $result = pager_query('SELECT cid, name, value, code, type, created, valid_from, valid_until, bulk FROM {uc_coupons} WHERE status = %d'. tablesort_sql($header), 20, 0, NULL, $view_type == 'inactive' ? 0 : 1);
  $rows = array();
  while ($row = db_fetch_object($result)) {
    $rows[] = array(
      theme('uc_coupon_actions', $row),
      check_plain($row->name),
      check_plain($row->code) . ($row->bulk ? '* '. t('(bulk)') : ''),
      $row->type == 'percentage' ? ((float)$row->value . '%') : uc_currency_format($row->value),
      format_date($row->created, 'custom', variable_get('uc_date_format_default', 'm/d/Y'), 0),
      $row->valid_from ? format_date($row->valid_from, 'custom', variable_get('uc_date_format_default', 'm/d/Y'), 0) : '-',
      $row->valid_until ? format_date($row->valid_until, 'custom', variable_get('uc_date_format_default', 'm/d/Y'), 0) : '-',
    );
  }

I wonder if anything is getting returned so I make sure the devel module is enabled and add dpm($rows)l after the while loop to see what value it has right now. (In this case I used dpm but if it got any more involved I would probably hook up xdebug so I could trace the functions and variables and get a better sense of what was going on.) After refreshing the page I see that the variable is an empty array so clearly nothing was being returned. Hmm... that is odd.

I could look in watchdog and see what the error message is but in this case the SQL query is fairly simple and doesn't take any variables except if it should return active or inactive coupons. To test it out I copy and past the query into my SQL browser and remove the brackets from the table names and set status = 1. When I run the query, I get the error "Unknown column 'valid_from' in 'field list'."

So now I know that the SQL query is not currently working because the column "valid_from" is not there. I do a quick inspection of the uc_coupons table and sure enough, the column doesn't exist. Maybe something is wrong with our schema. I open up the uc_coupon.install file in my text editor and look at uc_coupon_schema() to see if it matches what we've got in the database. I immediately notice that there are several discrepancies, the largest of which is that the valid_from column exists in the schema but not in the table. (I could also have used the schema module to see if they were the same but, hey, I'm old school.)

This usually means that the valid_from column was added somewhere in an update hook and wasn't in the original schema that we installed. Uc_coupons has 7 update hooks and the third (uc_coupon_update_6002) has the code where the valid_from column was added. When I check the following update hooks it appears they also haven't been run.

I run a database update just to be sure but it says everything is up to date. I even check the system table and it shows that uc_coupon is supposedly on schema 6006. At this point I realize that the updates haven't been run since 6001 so I reset the schema to 6001 and rerun the database updates. Sure enough, I didn't get any errors and everything seemed to work fine. (Please note that this was on my development copy, NOT on the live website. I replicated this on the live after I had fully tested everything.) Heading back to the coupons admin page everything now seemed to be working. Testing everything thoroughly showed that it was all fixed.

I'm still not sure how we ended up in the state where the system table was marked as having the updates run but they weren't run. While I've got the website but fixed for now, we still need to track down how this happened and if someone is fiddling with the live database in ways that they shouldn't be. Now I need to go install the schema module and find out if anything else is out of sync.

I hope this gives you some ideas on how to do bug tracking or at least gives you a glimpse into the world of web developers and what they go though when a bug is reported.

What steps do you use to track down bugs?

Photo Credit

Related posts: 

• drupal
• Show background image in jquery cycle on IE
• How to Find Drupal Modules

If you enjoy our content, please consider subscribing through RSS, so you can read our posts in your application of choice.

All tags: bugs drupal software Login or register to tag items

A preview of the Nuvole pre-conference training in Chicago

With the DrupalCon Chicago 2011 registrations going at full speed (early bird deadline expires tomorrow, hurry up!) we are receiving questions from people who are considering to sign-up for the Code-driven Development: Use Features Effectively pre-conference training by Nuvole.

As a general answer, here are 5 benefits in adopting a code-driven development workflow. The topics below are discussed in our relevant blog posts too and all will be covered at length, with practical examples, in our DrupalCon training.

1: Keeping track of all changes

When did a certain bit of configuration (such as a variable, or the allowed values for a content type) change? Was it a side-effect of something else or an intended modification? When all your site configuration is in code instead of being buried in a huge SQL dump, and you couple it with a proper version management tool, comparing site revisions is easy and effective.

2: Working in a distributed team

Why can't I work on setting up all content types and views for the Blog section of a website while someone else does the same for News? With no SQL dumps around, developers are free to experiment and restart from scratch without damaging each other's work. And they can work in a modular way on the same project in parallel (thus speeding up the development pace), even from separate countries!

3: Reusing components and settings

Isn't this site feature you are developing so similar to what you did in a previous project? Or this complex and tedious module configuration exactly the same across all your projects, so that you have resorted to write down the steps and repeat them manually? With a code-driven approach, replicating settings is just a matter of reusing the same, modular, extensible, components, avoiding tedious and error-prone manual work.

4: Updating production sites cleanly

What if a client asks for a complex modification on a production site? Do you need to take the site offline, dump the database for backup, repeat manually all the changes you had tested with an older dump and then finally put the site back online? Code-driven development allows you to use the same mechanism employed by the standard Drupal module updates, with virtually no downtime needed and a much cleaner log of changes.

5: Following the Drupal future

Worried that code-driven development could not be in the direction Drupal is taking? No need to worry! Some patches by the good guys at Development Seed made (or are making) their way into Drupal 7 and beyond, enabling a better support for a code-driven workflow out of the box. So code-driven development is not just cool, but future-proof too!

The DrupalCon training will be hands-on and will follow the model of our standard code-driven development training, with short presentations and practical assignments based on concrete examples, monitored by the Nuvole team.

Code-driven development is an approach that works in our case and addresses our needs; though, you may have different needs and thus find beneficial only selected portions of the workflow, for example; this is fine too!

drupal planetdrupaldrupal 7upgrade6.x7.x

This is now my first Drupal 6 to Drupal 7 upgrade. Not without hitches, but all in all not too painful. Hope you like the new look. As dedicated readers may know, I never stick with the same theme between major Drupal releases, more often than not because my previous theme was never ported and I can't be bothered to port it myself. The new theme is the rather pretty Corolla, which you can find here:
http://drupal.org/project/corolla

Here are some Drupal 7 upgrade notes and links others may find useful.

Firstly, the modules that are not ready/will never actually be ported:

CacheRouter
Content Access (but there is a patch: http://drupal.org/node/690610)
Messaging
Notifications
Nodewords
Node Images
TweetMeme
Twitter

So all of those I had to opt to live without for now (with the exception of Content Access, I applied the patch but I haven't had chance to verify it works yet because ACL has a critical bug - see below).

CacheRouter is a bit of a pain, as it was one of the easier ways to get some advanced caching going without needing to install anything in your operating system. You can read the full story here:
http://drupal.org/node/593238

I was using the 'filecache' engine which, as far as I can tell, has not yet been ported to use Drupal's new caching API, so your only option for simple caching right now is Boost. Which is still broken:
http://drupal.org/node/325813

The simplest currently available caching mechanism is probably the new Drupal 7 APC module, which is by slantview who made CacheRouter in the first instance:
http://drupal.org/project/apc

Install APC for PHP and install that module (don't forget to open README.txt and follow the steps) and you'll have some caching that isn't using the database and *should* be a significant performance gain on a weedy VM (like this one). It's not quite as straightforward as 'filecache' or Boost (which you just enable and tweak some settings and it's done) - you have to install other stuff - but it's pretty easy.

Next, some useful resources:

http://drupal.org/documentation/upgrade/6/7
http://drupal.org/node/570162

The former is all the upgrade steps you should need. The latter is cutting to the chase, the actual upgrade steps - it's linked to from the first one anyway.

So, following the steps I backed up everything, downloaded core and ran update.php over my old database. First thing you might see is this:

date_default_timezone_set() Timezone ID '3600' is invalid


Don't worry about it - it's harmless: http://drupal.org/node/1020748

So, that went ok. Next put the available Drupal 7 contrib modules and themes back in to sites/all (or wherever they were) and run update.php again to run their upgrade code.

If you use the Global Redirect module, you will have a problem at time of writing. Take a look at this:
http://drupal.org/node/861994

Not a biggy, but you'll have to patch/alter your install file for the upgrade to work. This will change with the next release, it seems.

And Rules currently has no upgrade path at all, so forget about that one for now. It's coming. Fago tells me [new window] clean installation should work though. Again, watch this space - an upgrade path is coming.

Another one that got me is a bug in the current Drupal 7 version of ACL. It's a show stopper right now, so use the dev snapshot, not the release, if you want to play with ACL - beta3 onwards should be safe(r):
http://drupal.org/node/1024114

Finally, once you're all set up you might see one or all of these little gotchas:

Expect to have to rebuild some Views (especially around Comments and user Profile fields). Many handlers are changed between Views 6.x-2 and Views 7.x-3, and are not automatically upgraded. This also results in some pretty horrible failures at first and freaks you out until you realise it's just missing Views handlers (*phew*).

No easy way around this. You have to go to Views UI and click your way through all the broken handlers, deleting the obsolete Drupal 6 handlers and creating new fields, sorts, relationships, etc. etc. with the new Drupal 7 handlers. This is a real pain. I'm hoping there'll be a Views updater at some point for your Views 2 views to clean up all the handlers. For now the upgrade path catches and repairs some instances, but nowhere near all.

Edit: Just found another issue after saving this node! All the tokens have changed and are not automatically upgraded it seems, so you'll need to go and edit your pathauto settings (and possibly other places where tokens are used) to make sure the correct values are being used.

Far less painful, you might see Filter module notices (I got one because of a filter provided by the Messaging module) - just re-save the Filter concerned (or delete it if it's not needed any more):
http://drupal.org/node/1018538

Finally, the new Fields replacement for CCK has some minor issues being resolved. You may (probably will, it seems) see notices like this:

Notice: Undefined index: required in field_default_form()
Notice: Undefined index: description in field_multiple_value_form()
Notice: Undefined index: description in field_multiple_value_form()

For now all you need to do is save the Manage Fields page of the content type and this goes away. Seems a coded fix is in the pipe:
http://drupal.org/node/931512#comment-3906836

One final observation. I seem to be experiencing overly aggressive menu caching with Drupal 7. After module removal links persist in admin, other menus are slow to update, seems there might be a small bug. And before anyone points at APC, it uses the caching API anyway and besides, I hadn't actually enabled it yet. One to watch...

That's all for now. Upgrade to Drupal 7! It's fun! (Kinda...)

There is currently severe flooding in Queensland Australia. An area twice the size of Texas is underwater. Entire homes are completely inundated. Bridges and cars have been washed away like toys. In Brisbane, airports are closed and the CBD has been closed down. There are at least 15 dead and more than 60 still missing.

QLDfloods.org is a Drupal 7 website set up by several members of the Australian Drupal community to provide information, track missing persons, find resources and people that need them (like beds), track damage and provide support. It was mentioned four times on CNN on Wednesday and multiple times on Australian national media.

The site builders are seeking help with Drupal 7 multiple-server configuration & infrastructure. Do you have expertise to help? Join #Drupal-AU on IRC, speak up in g.d.o/australia or contact Ryan Cross directly.

read more

Only a few weeks remain until DrupalCon Chicago, and if you haven't bought your ticket yet, now is the time to do so! The Early Bird Registration rate of $350 is still available, but not for long. Starting January 15, ticket prices will increase to $400, so act now and save!

Your ticket to DrupalCon Chicago covers admission to the conference, including the exhibitor open house on March 7, Drupal Means Business on March 10, and the post-conference sprints on March 11. Lunch is provided free of charge to all ticketholders on March 8, 9, and 10.

read more

The Site Building Extravaganza is running so I'm spending this week indoors, with a massive head cold setting up a membership site for all of my awesome registrants. While there are a lot of Drupal 7 contributed modules ready to go, some aren't quite there yet. On my plate for this week was importing all of the users from the Eventbrite registration site into my new Drupal site.

My first stop was User Import. This is the go-to module for importing users from a CSV (comma separated values) file. Sadly there's no D7 module yet. My next stop was Migrate. I know it can import users as well as content and it's just about the most awesome thing going, but sadly its self-documenting code was too much for my stuffed up sinuses. Finally I opted for the workaround of the century. Usability experts will marvel at the billion steps I took to accomplish this "simple" task.

1. Get the user names and email accounts into a CSV file. For this step I copied a PDF from Eventbrite into a spreadsheet, culled the columns I didn't want, and then exported the document as a CSV.
2. On your local workstation, install Drupal 6 and the User Import module. Note: I used User Import to ensure I had all unique usernames. This step wouldn't have been necessary if it weren't for the two "John Smith"s on my list.
3. Navigate to Administer › User management › User Imports. Click "Import" to start a new import.
4. Select your CSV file containing names and email addresses and click "next."
5. Choose the fields you would like to use for the user name, email, password and roles. (In my case I selected username and email only.) You may also want to adjust some of the settings for the username. For example: you can abbreviate first names, or include spaces between each name.
6. Scroll to the bottom of the page and click "test." A summary page will appear which displays the number of users that are "importable." Click "import" at the bottom of the summary to proceed.
7. Navigate to Administer › User management › Users to confirm your new user accounts have been created.
8. Now, using a MySQL client run the following MySQL command:
   mysql > select uid, name, pass, mail from users into outfile '/tmp/users.txt' fields terminated by ',' enclosed by '"' lines terminated by '\n';
9. A new file is created in the folder /tmp. Make sure the file name has the same name as the table that you are importing data into (i.e. "users"). Upload the text file to your server.
10. Log into the server and, from the command line, run the following command:
    $ mysqlimport user_import "./users.txt" -u YOUR_DATABASE_USERNAME -p --local --columns=uid,name,pass,mail --fields-enclosed-by='"' --fields-terminated-by=',' --lines-terminated-by='\n'
11. Finally, from a MySQL client on the server, run the following command to update the "created" timestamps for the new users:
    mysql > update users set created = unix_timestamp() where uid > 2;
12. The new user accounts are created, but inactive. Navigate to Administer › User management › Users. Select the new users and alter their roles (if needed). Then update their Status to "Active." This will also send a welcome email to the new users.

Tada! The new users have been imported into Drupal 7, and an email has been sent to each with login instructions.

If you've found an easier way to import user accounts into Drupal 7, please leave a comment in the box below. And if you'd like to learn how to build 12 Drupal sites from start-to-theme-to-finish, register now for the Design to Theme Drupal 7 Site Building Extravaganza.

Last month while conducting research for an upcoming report on Foundations and their use of Twitter, we started looking at what Content Management Systems (CMS) were running Foundations' websites. Our curiosity lead us to look at the top 885 Foundations with websites (based on endowment size), utilizing the Wappalyzer browser plug-in to determine how the sites were built and what technologies and services were in use.

A note on methodology: the list of Foundations and their website URLs came from the Glass Pockets transparency database. We used Wappalyzer during the month of December 2010 and looked only at the home page of Foundation. We were able to clearly identify the technology powering 478 of the 885 Foundation websites (54%).

Highlights

Here are some highlights from the tables below:

When we look at all the Foundations in our sample, Drupal is the front-runner among open source systems, but not by much. Wordpress trails in second, and just over half as many sites are running Joomla compared to Drupal. Most surprising to us is how few Foundation sites (just five) are running Plone.

When we look at the largest 500 Foundations, Drupal has double the Wordpress instances, and Joomla is right on the heels of Wordpress with just 3 fewer sites. In both samples, the proprietary web framework ASP.net powers the highest number of sites, however the lead is significantly smaller in the top 500 Foundation sample. Similarly, Dreamweaver drops from 10.51% of the full sample, remaining ahead of all the open source CMSs, to just 5% in the top 500 sample, a few points behind Drupal at 7.6%.

read more

This is the 6th installment of the Drupal 7 Line by Line series of articles.

Up to this point in the series I've covered index.php, the basics of the drupal_bootstrap function and have been working through Drupal's bootstrap phases. (Links to earlier articles in the series can be found at the end of this article).

In the last article I covered the Drupal database bootstrap. In that article I mentioned the following:

[when I wrote about] page cache bootstrap process I noted that unless your site is configured in a specific way, both the DRUPAL_BOOTSTRAP_DATABASE (phase 3) and DRUPAL_BOOTSTRAP_VARIABLES (phase 4) bootstrapping phases will be completed before the page cache (phase2) bootstrap process can itself finish. As a result, I am technically covering the database bootstrapping process out of order and am not literally following the line by line code execution of Drupal exactly. I hope you can forgive me.

Keeping that in mind, today I will cover phase 4 in the bootstrap process: DRUPAL_BOOTSTRAP_VARIABLES.

See Also:  Drupal 7 Line by Line Part 1 - Introduction Drupal 7 Line by Line Part 2 - drupal_bootstrap Drupal 7 Line by Line Part 3 - DRUPAL_BOOTSTRAP_CONFIGURATION Drupal 7 Line by Line Part 4 - DRUPAL_BOOTSTRAP_PAGE_CACHE Drupal 7 Line by Line Part 5 - DRUPAL_BOOTSTRAP_DATABASE

read more

Drupal 7.0 is out! And while we managed to get all the critical bugs fixed before it was released, there’s still a handful of major bugs that we weren’t able to fix. One of those bugs affects Drupal’s menus: Custom menus never receive an active trail. Oops.

Custom menus in Drupal 7.0

So if you are using a menu you created yourself with the “Add menu” link on the Menu administration page (admin/structure/menu), that menu won’t work properly in Drupal 7.0. If you visit any of the top-level pages in that menu, none of them will expand to show you their child links. This is really bad.

With last week's release of Drupal 7 and a steady stream of large enterprise sites continuing to migrate onto the content management platform, excitement abounds in the Drupal community. While celebrating all that is new, however, let's not forget about some of the early adopters that helped pave the way for all of the brand-name companies now hopping on board.

read more

The new module Webform MySQL Views can, combined with the Data module bring Webform submissions into Views.
For non-Drupalists, this seems just a long list of modules and/or nonsense words. For those of you who have tried to get Webform to work with Views, this is really good news. This 10 minute screencast shows you how to do it.

Pushing the Envelope on Web Mapping

This is the first in, hopefully, a many part series of short posts where I want to help highlight people in the Drupal community who I think are awesome, but might not get much spotlight. See below for more explanation.

The first person I want to highlight and say thank you to is Tom MacWright (tmcw). I have worked with him on the OpenLayers module for over a year now and he has been crucial in making the project what it is today and teaching me lots of things along the way. He also is very active in the overall web mapping space in Drupal and outside with such technologies as Mapnik and Mapbox.

Tom has recently lead the upgrade of the OpenLayers module to Drupal 7 (still alpha) and is very active in the issue queue. He has definitely picked up a lot my slack. Tom was also the driving factor behind getting the OpenLayers module on a Ctools architecture, which, though took me a while to see, has shown me the power of Ctools and the plugin goodness. This has lead me to starting a new project to build a common set of Ctools plugins for a general mapping architecture in Drupal.

Tom is also one of the leads at Dev Seed behind awesome work like live complex mapping in Drupal (TileLive), Maps on a Stick, TileMill, and so much more. Though a lot of this work is still not all that accessible to the average web developer, Tom and his colleagues have and are making huge strides towards a future where custom, open source web mapping is easy for us all. This is just awe inspiring and deserves a great big thank you; keep up the awesome work.

About These Posts

Drupal Highlights is a series of short posts saying a small token of appreciation to members in the Drupal community who might not otherwise get as much recognition as the more prominent contributors in the community do (though they totally deserve it as well). I think it is very important and strengthening for a community to showcase all the amazing things that happen in a community no matter how small. Please let me know if you want me to highlight someone you know.

read more

The conclusive article on a four part series on mobile sites in Drupal. Previous articles have discussed the groundwork, setup and code required for setting up a mobile site in Drupal. It’s now time to reflect on a few of the challenges thrown up and the best way forward. Gaps Given the above discussion there are a [...]

Previous articles have discussed the conceptual groundwork and setup of mobile sites in Drupal. It’s now time to look at a number of themes and modules which will help you in your endeavours. We’ll also cover a number of custom fixes which can be made to the HTML, CSS and JS in your site. Mobile themes Funnily [...]

A previous article covered some basic groundwork for mobile sites in Drupal. This article goes on to look at different ways to setup a mobile site in Drupal. It covers single site, multisite, single site with settings.php tweak and the Domain Access module. Caching strategies, redirect rules and other server side settings are also discussed. RESTful [...]

Recently I was involved in a project to convert a site into a mobile site for Drupal. During the process I had to overcome a number of problems – problems which pretty much everyone designing a mobile site will have to solve. This series of articles covers a bit of theory, the practical problems and [...]

This year, DrupalCon will again be providing room for "Birds of a Feather" sessions, or BoFs as they are more commonly known (it is commonly pronounced 'boff'). These sessions allow ad-hoc groups to meet and discuss a particular interest, how to scratch a particular itch, or really any topic that is on their mind. The format is unstructured and the topics are all over the map. This is what makes them great.

read more