Tuesday, January 29, 2008


PDO to turn PHP into closed software development?

These are my very private thoughts on what is going on with PDO.

First of all, having decent database support is becoming more and more important to PHP because there is hardly any PHP application left that does not have some kind of database backend. And for that exact reason, a few people started PDO some time ago.

Technically speaking, right now we are on the second PDO version - well starting to count unix-like from version 0.

The initial PDO developed by Sterling Hughes and me never made it into PHP. It was, however, the base for the initial version we later put into PHP, as it proved that our main design goals worked: unified API through a base class with different database back-ends connected through drivers, where drivers are separate extensions that sit on top of the main PDO extension. After some discussions at conferences - such as the International PHP Conference and LinuxTag - it was Wez Furlong who came up with the initial implementation of PDO version 1. It had a nicely worked-out callback infrastructure that could support a bunch of database backends. And, indeed, we got a few of those in short time.

So what we have right now is a more or less working PDO that suffers from the following:
So the main questions are: how do we attract more developers to PDO, and how do we improve PDO? And all this to boot without scaring away current developers.

Now thanks to Wez for the PDO 1 specification effort and talking to a bunch of database vendors, trying to integrate them into PDO development. Because only with a good documentation can we make any progress whatsoever, be it a full rewrite or just continuous development. Btw, my thanks also go to the countless people that wrote the PDO documentation.

However, I completely disagree with the way in which PDO 2 was started. The main reason is that throughout the preparation phase not a single developer had any clue whatsoever of what was going on - not even that something was going on at all! If that is not the case, then obviously those who did know were under some kind of NDA. It actually appears that there is a PDO development mailing list, which I was not aware of until two days ago (for PDO 0 we used my private mail server). And even if I had been aware of it, it still would not have been the PHP way of doing things - where the PHP way is doing development openly, or at least openly after a short startup time. Either way, to most people who read the announcement of a CLA in the PHP ACLs some time ago and the PDO 2 development proposal recently, this is a clear indication that the upcoming PDO version won't be open source. Because open means open and the PHP community in particular has always been especially open. Even though, as we recently discussed again, we might have a reason to slow down traffic on internals@, in general we all profited from our openness.

From my very own perspective, I do not see any reason to change the ways in which we are dealing with things. That includes, first of all, that we discuss things openly. And second of all, that we have everything that is in PHP core under the PHP License - apart from bundled libraries, including the Zend engine and TSRM, of course. We btw discussed some time ago that everything in core in fact should be under the PHP License. And so far we do not have a single exception. Instead, we are continuing to replace parts of PHP that have a different license with our own rewrites/re-implementations. If we were to change the way PHP core components are developed or licensed, we would not only scare people away, but would also run the risk of this getting completely out of control.

But when we do not change anything, we do not make any progress. So let's all rethink that.

So we need a better PDO core and better PDO drivers. And that means we need to come up with a better callback infrastructure between main PDO and its various drivers. And that requires some insight in the actual databases, where there are two problems. First, some vendors would only be willing to give us PHP developers insight if we or our companies signed a NDA. And second, the vendors do not usually talk about their internals unless they absolutely have to. And if they do, then this traditionally would be done under a NDA as well. But open source development and NDAs do not really mix. So the NDA way is out.

Now something between completely open and completely closed is (and the Apache Software Foundation does this with quite some success) to use CLAs.

Since I am absolutely against any change of PHP core development because any change might spread out to larger parts of PHP development, let's only deal with PDO here.

That said, PDO main belongs in PHP core, which means that it has to be done under the PHP License which is pretty much incompatible with a CLA. But we can still develop PDO drivers in PECL. And there we can even allow different licenses, at least as long as they are compatible to our OSI approved license. Actually, we already have PECL extensions where people can only contribute after agreeing to some special terms.

At this point we could live happily ever after - if only there weren't the issue of developing a high-performing callback infrastructure. Oh, and we still would need to find people to work on PDO main.

So how do we come up with working specs and people who can contribute?

Right now, we have people contributing to PHP on a regular basis from companies like Google, IBM, MySQL, Oracle, Yahoo, Zend and a bunch of various others - be it large companies, database companies for the PDO matters, established open source companies, or small private companies to freelancers like you and I once were.

Clearly companies can find ways to work on open source without CLAs. And if any company has a problem discussing the protocol of callbacks without having everyone else first sign a CLA, then I can only say that it is their very own problem and potential market disadvantage. Pretty polemic answer, sure. But PHP has gained a lot of momentum in the previous few years and we, the PHP contributors, have accomplished that with a hell lot of effort. Private efforts for the most (and some paid efforts as well). Each and every one of us. So I do not feel the slightest temptation whatsoever, to give even the tiniest piece to any closed company. Plus having to sign a CLA might not be possible to all people working on PDO so far. And with that in mind, I would rather live with having an imperfect callback infrastructure in PDO than losing a list of respected open source contributors.

Sorry to say this, but I do not want a CLA in any part of PHP core.

That of course means that we still have to talk to database vendors. And, guess what, we probably welcome any kind of contribution. Just as we have always gladly accepted any kind of contribution. And maybe the actual problem is starting up a discourse with the vendors. Maybe the problem here is that there is currently no PHP entity. The only thing close to a PHP entity is the so-called PHP Group. But other than having a mail address that goes to some people mentioned on the credits page, the PHP Group does not appear to be doing much of anything. So maybe what we really need to do is fix this situation and recreate the PHP Group. Restart it with people that actually work on PHP and know the internals. Maybe just like PEAR Group by voting; maybe like the Apache Software Foundation; maybe in a completely different way.

Parting words: If you are wondering what PDO 2 is, well so am I. And how is this open source development?

Blogged with Flock


The "countless contributors" are actually almost exclusively Dan Scott (who worked for IBM at the time). Some other people have made changes, but Dan was the main contributor to the PDO docs.

Dunno about PDO and maybe it's my ASF background but I can't see why a CLA is seen as a disadvantage for an open source project.

At the ASF CLAs have always been seen as a protection for the contributor as well as the project and its users - nothing more.

Signing a CLA shouldn't mean that you give up any of your rights in regard to a contribution (in contrast to NDAs or FOU restrictions), see the ASF CLA for an example.

So I don't understand why you say that the PHP license is incompatible with CLAs or why companies prefer to sign them before working on open source. They certainly like to "see" them before *consuming* open source since they then can be reasonably sure that they are on the safe side in regard to IP...

The reference to closed-ness in relation to CLAs also completely baffles me... :)

Can you post a link to the PHP-/PDO-specific CLA? It sounds that it's more an NDA instead of a classic CLA...

I also read through the announcement trying to figure out what it was all about. My conclusion is that the current implementation is not suitable for some parties and they want to re-build it with open discussions to make sure it's all fine. Summary: They have no idea what they are about to do, but that's why they do it.

I don't know so much about CLAs, but reading the announcement still gave me a good feeling about how things were going. It didn't look that bad.
Sean, you are absolutely right, when writing about the PDO documentation, I really should have mentioned Dan Scott separately. It is just because of his efforts that we got those docs.
This post has been removed by the author.
@erik: http://www.php.net/~wez/pdo/

Generally the perception in the PHP community is that we fared well without a CLA, that a CLA is mostly only relevant for people in the states and therefore offers no benefits for non US contributors and that a CLA adds a legal hurdle for people to contribute.

So what might happen goes in the direction of what Marcus mentioned. The core and specs of PDO are not CLA'ed and the vendors can write their drivers outside of core.
I don't think a NDA was involved at all in the initial talks with the DB vendors. The developers involved probably felt that it would be best to push the vendors to something that may be agreeable to the community at large before bringing in the community, rather than letting one vendor's initial request for a full NDA or closed driver kill any further conversation right off (and judging from the reaction this has gotten, that most certainly would have happened). They did come to the larger community before going forward though. They aren't making a private decision that everyone needs to live with; they are asking for input.

A CLA does not make the code closed source. It doesn't keep you from viewing the source code (no matter what some would say), and it doesn't keep you from making modifications to the source code. The only restriction is on who can commit to the main branch. This is really a protection for the users and developers more than it is a protection for the vendors themselves. This will keep MS or Oracle, etc. from having one of their developers "surreptitiously" insert IP protected code into the main branch and make the PHP community responsible for it.

As for other companies contributing code, you have chosen some pretty bad examples: Google, Yahoo and probably IBM aren't contributing code critical to their business model or related to their IPs. MySQL and Zend are both companies built entirely around open source software and thus don't have anything to lose (and a lot to gain) from contributing. Oracle is a good example, and that is why I don't think they were one of the companies pushing for the CLA.

My main concern, and why I am still on the fence about this CLA, is bug fixes from the community and how they will be treated. This is something that needs to be thoroughly addressed in the CLA before I can get behind it fully.
Hi Marcus, I agree with Erik Abele that a CLA shouldn't be a disadvantage.

Personally, I don't care whether PHP allows PDO to use a CLA or not. But I'd like to help answer some questions to remove some of the mystery. Then you'll have more information on which to base your opinion.

It was assumed by all vendors that the development of PDOv2 would be open. No one ever suggested that an NDA would be appropriate. All these vendors are regular contributors to other OSS projects. They they value and respect the goals of free software.

The intent of the CLA is very similar to the traditions used in the PHP project. That is, you are giving your code for free, and it's yours to give. It's just more clear and formal because it's in writing. I think if anything, the proposed CLA is milder than the Apache CLA.

No technical plans have been developed, and no PDOv2 spec has been written. That's waiting for the legal and admin stuff to get settled so we know who is participating (besides the PHP community of course).

The one technical thing that was discussed was adding some metadata support to PDO. But that was also tabled until everything else gets settled.

"Settled" includes approval by the PHP community. Everyone in the group of vendors knew this was a requirement. They just wanted to deliver a reasonable fully-baked proposal for discussion.

Mostly the meetings were about figuring out *if* these vendors could work together, and if so, how. This was not about technology, it was about the relationships between these vendors.

The private mailing list was used for scheduling discussions between the vendors.

Even though some of the vendors have contributed to PDO in the past, some said that it's *not* assumed that they would be permitted to contribute to PDO in the future without a CLA in place.

I wrote in my blog about my experience using a CLA in the Zend Framework project. The CLA for PDO might be managed differently, but a lot of what I wrote still applies. You're welcome to read what I wrote:
Post a Comment

Links to this post:

Create a Link

<< Home

This page is powered by Blogger. Isn't yours?

Subscribe to Posts [Atom]