DrupalFire!

Davy Van Den Bremt (via DrupalFire)

For one of our clients, we are running a Drupal site with about a millions of nodes. Before launch, those nodes are imported from another database and then indexed into Apache SOLR. The total time to index all of these nodes in an empty SOLR instance is measured in days rather than hours or minutes.

A bit too long to do this import regularly. So me and my (XDebug) profiler delved into the Apache SOLR module code to look where we could scrape of a few hours/days of the execution time.

Seemed like in our case, there were 3 components responsible for a large share of the execution time. Let's have a look.

BTW. We are using the latest dev build of version 2 of the Apache SOLR module.

read more

Lullabot (via DrupalFire)

Drupalcon Copenhagen slides and a video!

I had a great time at Drupalcon Copenhagen! Thanks to everyone who made it happen. I did one presentation this time around, "The Command Line is your friend." It covered the basic commands for getting around and doing things, most of which are covered in more detail in the Command Line Basics video series. One thing that was new and that I ended up not having time to get to in Copenhagen was showing how to install Drupal from the command line. A number of people expressed interest in seeing that part, so I promised I'd make a video of it, and now I've gotten it done. I'm attaching the slides from the presentation here as well, so please have some fun playing around on the command line.

• Archive.org video of the live presentation
• My slides as a PDF
• The new Command Line Basics: Install Drupal video

Lullabot (via DrupalFire)

How to create presentations that entertain & inform.

I had the opportunity to present to a local organization in Salt Lake City on the techniques Lullabot uses for making kick ass presentations and how we streamline our presentations so they're reusable amongst our teaching staff. Watch the video below and download the slides.

• Ditch PowerPoint. PowerPoint prefers boredom, repetition and information fatigue.
• Do a cold open and talk about something relevant: the weather, the setting, world news.
• Kill the bullet points. People don't retain bullet points. They retain the story.
• Don't apologize for a demonstration not working as expected. Either figure it out together with the audience or simply move on.
• Simple, but not simplistic. You're audience is smart. A one-by-one reveal of bullet points is simplistic. A large photo with a brief message is simple (and brilliant).
• When creating reusable slides, use learning objectives and outlines to supplement the presentation while leaving room for each presenter to share their stories.
• Make it fun! Connect the presentation to what you're passionate about.

Development Seed (via DrupalFire)

Plans for the Aegir 1.0 release and how we'll get there

Last week at DrupalCon Copenhagen Antoine Beaupre and I shared our plans for the 1.0 release of the Aegir hosting system. After the 0.4 release that we are currently working on, the project will start working towards a final 1.0 build, rather than head towards 0.5.

Our goal for Aegir has been to implement a hosting system with a stable documented API that provides a solid foundation for integration of other services. Once we hit the 0.4 release, we will have the infrastructure in place to support this. With our key goal met, we've begun determining our must have features for a 1.0 release.

Going Drupal 7

Aegir 1.0 will be built on Drupal 7. Due to the commoditized nature of hosting infrastructure, we felt it would be impractical to have a stable release that requires additional investment whenever the API changes during development. Therefore our primary goal is to rewrite Aegir's user interface for Drupal 7 and release it as early as possible in the Drupal 7 lifecycle.

With this we hope to ensure that custom and contributed extensions to Aegir built for the 1.0 release will stay viable for as long as building Drupal 7 sites remains viable, allowing your investment in the platform to remain usable for upwards of two years.

New features

After focusing heavily on Aegir's backend in previous releases, we will be turn our attention to the user interface for the 1.0 release. We want this release to be the most lickable version of Aegir yet, with many user interface and workflow improvements.

We plan to implement new functionality to ease and automate issues surrounding 'Development to Staging to Production' workflow problems felt by our users. We'll further integrate with Drush make, allowing Aegir to subscribe to new releases and automatically download them when a new one becomes available. This will allow you to migrate sites to the latest version literally with a click of a button.

The 1.0 release will also focus heavily on interoperability. We will rebuild the hostmaster install profile to Kit specification, replace custom listing code with Views (something which is already in progress), and integrate more closely with Features, Context, and Spaces. This will simplify our code base and allow for useful functionality like the ability to create SSL sites per client and possibly integrate Open Atrium features with Aegir.

We are open to feature suggestions from our users for the 1.0 release. If you have a request that would improve your experience with Aegir, please send it to @aegirproject.

Finalizing the 0.4 release

We are close to wrapping up the 0.4 release cycle. Once we have completed DNS slave server support (master support already works), we will be feature complete and will start releasing betas in preparation for completing 0.4. We hope to have this functionality completed in the next few weeks.

Slides from the DrupalCon session Aegir: One Drupal to Rule them All are up, and we'll post a link to a video of the session here once it's online.

CivicActions (via DrupalFire)

CivicActions has been working with Google's “Make the Web Faster” project team to make some (last minute) improvements that make Drupal 7 faster.

read more

Matthew Saunders (via DrupalFire)

Richard Jones of iKos gave a presentation on simplifying and streamlining the development of Drupal sites. His team observed that they were creating the same basic sites over and over again. To this end, rather than starting from scratch each and every time, the team created basic sites to start from and rapidly built out the basic site for a client in several weeks. This allows the client to start entering data early in the process - which means come theming time, you are working against real client data. This is a plus when you are trying to achieve the results your client is seeking. In the iKos model, design comes last and is somewhat subject to the functionality that has been developed.

iKos set the the goal to reduce Drupal development time by 80% - to radically reduced the repeated effort that the company engaged project after project. They created a model of a site "startpoint" that begins each new client effort.

read more

Gábor Hojtsy (via DrupalFire)

It is that phase of my life! I'm just turning 30 in a month, working with Drupal for 7 years and just had my third Acquia anniversary a week ago. Time to look back and evaluate how things went, all the good and bad things; even better if the wisdom can be shared with others. This was part of my thinking when I submitted the session titled "Come for the software, stay for the community" for Drupalcon Copenhagen. I was interested to distill and share how Drupal came to be as unstoppable as it is, what core values lie behind it, so someone coming fresh can understand and integrate with these.

When Dries Buytaert started Drupal he made a few key decisions which launched the project and kept being governing principles ever since. First of all he decided to make it free and open source, and release it under the GPL. The choice of one single license helps you use all the Drupal components together without the requirement to consult lawyers. Also, the choice of GPL in particular ensures that derivative works are distributed in the open as well.

Using PHP as the base technology was probably the best decision to help proliferation. PHP is not only the most installed backend software (thanks to all kinds of server side software), but its also easy to learn and jump into. While this can and did end up with some projects that gave PHP a bad name, it also opened the door wide to contributors. Having numerous contributors encouraged the tinkering, technology experimentation spirit that defined Drupal from the gate. What's more, Drupal was an integration platform from the beginning. Even Drupal 3 (in 2001) included RSS generation and consumption, a distributed and automated website directory and tagging. Drupal 4 (in 2002) added Jabber user account integration, XML-RPC services support and Drupal shared authentication. See, Drupal had lots of the base features we praise it for even 8-9 years ago. Woo!

Having a technologically advanced system that people can contribute to easily is not enough. You want to build a culture of building on top rather than forking to avoid the bad fate of many other systems grown up through the years. Establishing the hook system on the API level and the node system on the content handling level formed a good foundation to let people tinker even with the innards of Drupal without ever needing to fork.

Building a big home for all these contributions was the next major idea that distinguished Drupal from many other projects. Drupal itself was built on a sharing cultural mindset and drupal.org was set up for all kinds of contributors to follow suit: release code, designs, tools under the same licenses with the same issue handling system, etc. This helped people move between projects of their interest, contribute to each other's work. Having a strong central base versus everybody's own little kingdoms around the internet jumpstarted the large Drupal ecosystem. In 2009, we had above 2200 new projects started on drupal.org, that is above 6 per day.

Keeping the community all together, Drupal could embrace a culture of change as well. Drupal core could be rearchitected with every major release and have tools to help contributors keep up with the flow. This way Drupal can be at the forefront of innovation both in core itself but even more in contributed modules and themes. As Drupal itself keeps improving, contributed modules are forced to rethink their approaches, purpose, and get better in the process. Old and cumbersome solutions vanish while new ones take their place. Developers and users both come out empowered at the end.

This all provided with good ways to do power distribution without appointment. Everybody can be the king (AKA well respected lead) of their modules, themes, documentation sections, translation teams, and so on. There is no need to appoint people to positions per say and possibly face issues when people become inactive. The centralized project and code management space gave way for power transfer when needed, making it easy to hand off maintainership and continue work without breaking history.

All-in-all it is both Dries' forward thinking launch of Drupal and the bright minds who carried on and continued following the same basic principles for almost a decade. This is why we consider Drupalcons where people can meet and drupal.org where people do Drupal work our most precious possessions. This is why the Drupal Association puts money first and foremost into making drupal.org a better place for both newcomers (with the redesign) and for developers (with the git migration) as well putting on Drupalcons around the world. While looking for the next big thing is always on our mid, we managed to keep our values and build a happy community on the way. Win!

Ps. The new Drupal Code of Conduct was just published on drupal.org which might help you understand some of the basic human values we strive for.

read more

Matthew Saunders (via DrupalFire)

Ryan did a session on the new version of the D7 Commerce Module. He was the original writer of the Ubercart package - a main stay of ecommerce sites in Drupal 6. His self deprecating humour at the beginning of the session was charming as Ubercart was his first segue into many aspects of Drupal and it has become the defacto leader of the pack for Drupal ecommerce sites.

His new offering is looking very slick, making use of fields in core for example, and while Ryan tells us it isn't quite ready for primetime (he's still building out features) it looked pretty good.

Video from this session is below in seven sections. The original first video was only 4 seconds long.

read more

Kristof De Jaeger (via DrupalFire)

I've been promising benchmarks for the Display Suite module after every presentation I gave so far. It took me a while to get a good setup but now it's here. I've used the demo site as a start, so there are a lot of modules enabled for this test. Views, panels, fivestar, heartbeat, comment, taxonomy, location, gmap, imagecache are the most important ones since they all integrate with the ecosphere of Display Suite modules.

I added a new content type called 'benchmark' and added 14 CCK fields to it: 4 textfields, 4 textareas, 2 images, 2 filefields, 1 node reference and 1 user reference. It also has a title, body, 2 taxonomy fields, a fivestar widget and a couple of comments.

Depending on the test, the complete set of modules integrating with Display suite are enabled or disabled. These include ds, ds_ui, cd, hds, nd, nd_cck, nd_search, nd_fivestar, nd_location, nd_switch_bm, ucd, ud and vd. You gotta love small project names right ?
<!--break-->

Desktop

The first test was ran on my Fedora Core 13 desktop - Intel Core Quad, 2 GHz, 2MB RAM with php 5.2.13 and eAccelerator - ab sending 100 requests with 5 concurrent users on a single node and page caching disabled.

Without Display Suite
PHP 4.91 MB
Requests per second:    36.18 [#/sec] (mean)
Time per request:       138.202 [ms] (mean)
Time per request:       27.640 [ms] (mean, across all concurrent requests)

Build mode blocked
PHP 5.31 MB
Requests per second:    34.68 [#/sec] (mean)
Time per request:       144.174 [ms] (mean)
Time per request:       28.835 [ms] (mean, across all concurrent requests)

With Display Suite
PHP 5.33 MB
Requests per second:    34.51 [#/sec] (mean)
Time per request:       144.876 [ms] (mean)
Time per request:       28.975 [ms] (mean, across all concurrent requests)

Server

The second test was ran on a CentOS 5.3 - 2 x Intel Core Quad, 2,6 GHz, 8MB RAM with php 5.2.11 and eAccelerator - ab sending 100 requests from my pc to the external server with 5 concurrent users on a single node and page caching disabled.

Without Display Suite
PHP 4.25 MB
Requests per second:    26.91 [#/sec] (mean)
Time per request:       185.775 [ms] (mean)
Time per request:       37.155 [ms] (mean, across all concurrent requests)

Build mode blocked
PHP 4.61 MB
Requests per second:    26.76 [#/sec] (mean)
Time per request:       187.072 [ms] (mean)
Time per request:       37.514 [ms] (mean, across all concurrent requests)

With Display Suite
PHP 4.63 MB
Requests per second:    26.65 [#/sec] (mean)
Time per request:       187.642 [ms] (mean)
Time per request:       37.828 [ms] (mean, across all concurrent requests))

I knew that enabling modules would cause the memory of PHP to grow on both my desktop and the server. The server however behaves better in terms of keeping the requests per second steady per setup, so that's the real good news. I'll probably run some more in the future to see how it behaves in other situations, but I'm glad that I've got some results to show!

You may lose a few microseconds, but win days of maintainability!

Matthew Saunders (via DrupalFire)

Many thanks to everyone who made Drupalcon CPH happen. The final session is a suitable (if silly) ending to what was a very productive week for me. Lots of meetings, reconnecting with old friends, and sessions.

The greatest silliness and fun in the session were the Kitten Killers - you can see just that piece of the presentation here (again and again and again).

If you just want to see the final session from beginning to end - I've posted the entire session below. The second to last is the Kitten Killers.

read more

Matthew Saunders (via DrupalFire)

Have we replaced the Drupal Song? At the end of the final session, the Killers played a new song, "I Can Be Your Module, You Can Be My Theme". It was a fun and silly way to end the main part of the conference and segue into the sprints.

The song was fun and sounds like it would be really easy to sing again and again and again.

Here's hoping it has been placed in the public domain.

read more

John Forsythe (via DrupalFire)

Every day, millions of people use Google Image Search to find pictures, products, and people. If you're using Drupal, chances are you're not getting any of this traffic.

Here's why:

Drupal's robots.txt file contains a major mistake. Amazingly, the mistake has been there for years, and very few people seem to know about it.

read more

Lullabot (via DrupalFire)

Save 25% On All Lullabot Learning Series Videos!

Use coupon code: SCHOOL25

The school year is starting up, DrupalCon CPH is over and it's time to learn Drupal! For those of you who've been waiting for the perfect time to grab your copy of one of our awesome Learning Series videos, now's your chance to save big. Now through Friday, September 3rd you'll aget 25 percent off your purchase of any physical or downloadable product in the Lullabot store by using the coupon code SCHOOL25. That includes ALL of our Lullabot Learning Series videos and video bundles!

To get this special savings, just use the coupon code SCHOOL25 at checkout and place your order before the sale ends! Tell your friends and share it with others, there are no limits on this coupon code, but it ends Friday, September 3rd.

Visit store.lullabot.com (or click the "store" link in the menu above) and shop for whatever's been tickling your fancy.

What others are saying about Lullabot's Learning Series Videos

Just finished watching Advanced Theming video. It was exactly what I needed to move ahead with my current project. Completely demystified template.php, and Forms API is now malleable. You guys are the pros, you rock!
– waj194 (via Get Satisfaction)

Gábor Hojtsy (via DrupalFire)

Anders Høeg Nissen from Harddisken, the P1 Danish Radio show was out at Drupalcon Copenhagen to report and interview people about Drupal and just generally spread the news. P1 is part of one of the oldest and largest media empires in Denmark, its parent company was founded in 1925 as a public service organization.

Dries Buytaert and Angie Byron were on the show being interviewed on their thinking of Drupal, and how they manage the flood of people coming with Drupal 7. I was interviewed to share some of the ideas behind my session titled "Come for the software, stay for the community - how Drupal improves and evolves". The radio host was interested in what I think are major drivers in Drupal's thriving community and how do we make it work. We got some of our thinking translated to Danish even.

I unfortunately don't know Danish (like probably most of my readers). If you know, a transcription / translation would be useful, thank you. In the meantime, you can listen to the show (MP3) mostly in Danish. (The first 25 minutes cover Drupal).

read more

Gábor Hojtsy (via DrupalFire)

As Drupal events grow around the world, more and more people find meetups and conferences closer to themselves. However, traveling to bigger events like Drupalcons can still be a financial problem for many. One of the solutions for this is couch surfing, where you could take a couch from someone who has it available in the host city for an event. Of course sleeping at an unknown person's place can be problematic. However, if we consider you already go for a Drupal event, and drupal.org has a profile for people with attached data on their activity, it can form as a reputation system. At Drupalcon Copenhagen, we discussed that maybe it would even be possible to build this tool as a group on groups.drupal.org (but definition of what is exactly needed is best to be done first).

I don't have the opportunity to help and build this tool but thought throwing the idea out would be useful for others to brainstorm and possibly help the community by making it work. We already had a nifty slogan for it when module maintainers couch surf: "I'm using your APIs, may I use your couch as well?"

read more

Development Seed (via DrupalFire)

Aegir's new home, release updates, and DrupalCon

The Aegir Project has just launched its new home - aegirproject.org. Right now this site is just a placeholder, but we'd like to turn this space into a resource for documentation, release announcements, and community discussion. We'd love to hear about your ideas for what you would like to see on the site - please send ideas to @aegirproject.

Development on Aegir has been advancing quickly this summer. Earlier this month we put out the Aegir 0.4 Alpha 9 release, which added features like multi-server support, a vastly simplified installation, and support for sites encrypted with SSL to the system, and we just announced the 0.4 Alpha 12 release, which contains bug fixes and usability improvements. The last major feature missing before we can consider Aegir 0.4 feature complete and start the beta cycle is the ability to host DNS for all your sites. This is a requirement to allow us to move sites between servers successfully.

I'll be presenting on Aegir at DrupalCon Copenhagen tomorrow, August 26th, at 1:30 pm in the Trellon Room with Antoine Beaupré. We'll walk through Aegir and show off the new features that have been added over the last year, especially some of the multi-server magic like how to use Aegir to add additional web and database servers and migrate and clone sites between several servers. We'll also talk about our future plans for the project and share best practices around how to lessen the impact of the development -> staging -> production workflow problem by making use of tools such as Drush make and Features.

For those who have already consumed the Aegir koolaid (or beer, since Aegir, the Norse god of the ocean, is known for his love of beer), we will have a BoF at 4:00 pm tomorrow and after that, meet in the lobby to head out to our traditional Aegir dinner. This is always a great time to hang out in person and get to know the team better.

Development Seed (via DrupalFire)

Open Atrium goes Kit compliant

Yesterday we released Open Atrium 1.0 Beta 8 (aka "Seatbelts"). This release makes some of the biggest changes to Open Atrium since the Beta 4 release last winter, focusing on compatibility, accessibility, and stability.

Compatibility

Eight features in Open Atrium are now Kit compliant. Kit compliance pushes Open Atrium to a new level of modularity, allowing any of the eight features to be used in other distributions or your own projects with fewer dependencies and less hassle than before. Changes related to Kit have also improved the compatibility of Open Atrium on several other fronts:

• The Seed module that provided a grab-bag of UI tweaks and utilities has been deprecated in favor of several projects that accomplish the same jobs and better. ImageCache Profiles now provides the user picture thumbnails, Node form columns makes it easy to rearrange content forms, and iTweak Upload streamlines the attachment UI.
• A pattern of optional enhancement has been adopted across Open Atrium features, allowing you to use them in their simplest incarnations and then progressively add on niceties like Organic Groups and Spaces integration, Relative date display, and Crayon colored popups/condensed displays in lists.

Caleb G (via DrupalFire)

First of all, I dislike writing this and posting it, only less than I dislike the need for it. With that said....

...self-appointed sexism police in the Drupal community, how dare you. Seriously - berating and making a public spectacle of those comments from Dries. Petty, petty. Several years ago, I was personally witness to what happened to the unsuspecting persons who had ended up having their comments drawn out into a hyper-dramatic indictment at http://rocktreesky.com/reactin.... It made my stomach sick then and a few other people that I remember too, but when the thought police are running around affixing Scarlet S's on people no one really wants to stick their neck out to far, lest it gets chopped off with the rest.

That was then though, and the small number of folks who seem to have standards which would require them to hate the larger percentage of men on the planet (hey it's fine to be gay, but not everyone is) were not picking on the Dries Buytaert, someone who in my book has only ever shown the best character, class, and motives (and I can't forget - also started the open source project Drupal which has provided me a great living as well as the people looking to hang him for his innocent words).

How dare you.

You set your sights too high this time. It's an easy mark when you try and question the words of someone with class as low as yours, but this time you've questioned the class of someone that you should feel ashamed to mention your own name with. If you have anything else to say about Dries, just say, "thanks".

For the rest of the people who aren't part of the thought police, I hope this galvanizes you all to stand up for what is right and to insist on community where one can speak openly without fear of having any random comment they might utter held up as an example of the "evils of sexism".

Finally, a follow up about my comment above concerning 'it is fine to be gay, but not everyone is'. In my mind, there is something especially not right when some of the loudest voices in this 'sexism' crusade also happen to be lesbians. And yes, in this case one's sexual preference is very relevant. If I were a gay man I would not be running around telling all the women in the world how they "should behave and speak".

Bottom line, if someone says something which was not meant to offend you and you get offended anyway and start telling them everything they need to change about themselves, then maybe it is not they who needs to alter their behavior.

read more

Aaron Winborn (via DrupalFire)

As KarenS predicted, there have been many Drupal-free nights from me this summer, while my household adjusts to its newest member. Though thanks to Advomatic, my days have continued to be filled with Drupal.

For instance, I've been hard at work on Node Reference / Embed Media Browser (nrembrowser), the perhaps unfortunately named, but descriptive, module that does just what it says. I did a session for it in an earlier incarnation at Drupal Dojo, and have banged at it incessantly, until it's now just about ready for prime time! As I said in the session, that project is, I believe, a reasonable alternative for folks who want Media-like functionality for Drupal 6.

What makes it viable is its integration with WYSIWYG and Styles. Styles is a brainstorm I had during the development of Media, in which I tackled how to display wildly differing data (such as photos and videos) with a single field formatter. I've been refining the model in version 2 of the Drupal 6 version, where I've moved it into a class structure and made lots of improvements. Coming soon to Drupal 7...

And then there's Views Slideshow: Galleria, which makes it easy to create a Galleria slideshow in Drupal. And some refactoring of Embedded Media Field, version 3, in which we begin the migration to Drupal 7's Media, and bring some of its magic (of a unified storage system) back to 6...

But what I've been dying to tell you all is about my few, but precious, Drupal nights. Though it's gone from 12 hours or more a week to three, in the wee hours of the morning at that, I've still been banging away. However, I've changed directions entirely, following another passion of mine: games! And no, not playing them, making them... I've said too much already, as it's not quite ready for release. But suffice to say, I think it'll be fun, and of course, it'll be available through the good ol' GPL...

read more

Károly Négyesi (via DrupalFire)

The Drupal community is remarkably not sexist and yet... Yesterday, Dries' keynote, alas, had a sex scandal slide which probably just wanted to be a (rather tactless and tasteless) joke but it had rather sexist undertones which have managed to piss off a number of women. But, my frustration is not just from the keynote but the reaction, let me quote two tweets here: "Woman on stage at #drupalcon, Dries can have his sextape in 2010." and even more importantly, from someone who retweeted it and got called out for it "Don't take it so seriously. It was a joke, right?" Now, head over to http://rocktreesky.com/reacting-sexism and you can see that three years ago (exact to day) the documentation team leader of Drupal have written exactly, word-for-word about this. It is absolutely unacceptable behaviour and not a joke to degrade women who have made DrupalCon possible for you to sex objects. As Addi noted in her linked blogpost, sexism won't stop just because we write about it, however, I hope this accident will actually make the community even more refusing about sexism and it will be unnecessary to write about it for at least another three years and we can work together on releasing an awesome Drupal 7, 8...

read more