staying neutral

These are my very private thoughts on what is going on with PDO.

First of all, having decent database support is becoming more and more important to PHP because there is hardly any PHP application left that does not have some kind of database backend. And for that exact reason, a few people started PDO some time ago.

Technically speaking, right now we are on the second PDO version - well starting to count unix-like from version 0.

The initial PDO developed by Sterling Hughes and me never made it into PHP. It was, however, the base for the initial version we later put into PHP, as it proved that our main design goals worked: unified API through a base class with different database back-ends connected through drivers, where drivers are separate extensions that sit on top of the main PDO extension. After some discussions at conferences - such as the International PHP Conference and LinuxTag - it was Wez Furlong who came up with the initial implementation of PDO version 1. It had a nicely worked-out callback infrastructure that could support a bunch of database backends. And, indeed, we got a few of those in short time.

So what we have right now is a more or less working PDO that suffers from the following:

• Some databases have suboptimal drivers (often slower than native extensions).


• No support from database vendors to help improve drivers (for various reasons).


• A maintainance nightmare, where PDO in different PHP branches is completely out of synch.


• Limited functionality in PDO (like missing metadata support).


• Missing developers for the main PDO extension.

So the main questions are: how do we attract more developers to PDO, and how do we improve PDO? And all this to boot without scaring away current developers.

Now thanks to Wez for the PDO 1 specification effort and talking to a bunch of database vendors, trying to integrate them into PDO development. Because only with a good documentation can we make any progress whatsoever, be it a full rewrite or just continuous development. Btw, my thanks also go to the countless people that wrote the PDO documentation.

However, I completely disagree with the way in which PDO 2 was started. The main reason is that throughout the preparation phase not a single developer had any clue whatsoever of what was going on - not even that something was going on at all! If that is not the case, then obviously those who did know were under some kind of NDA. It actually appears that there is a PDO development mailing list, which I was not aware of until two days ago (for PDO 0 we used my private mail server). And even if I had been aware of it, it still would not have been the PHP way of doing things - where the PHP way is doing development openly, or at least openly after a short startup time. Either way, to most people who read the announcement of a CLA in the PHP ACLs some time ago and the PDO 2 development proposal recently, this is a clear indication that the upcoming PDO version won't be open source. Because open means open and the PHP community in particular has always been especially open. Even though, as we recently discussed again, we might have a reason to slow down traffic on internals@, in general we all profited from our openness.

From my very own perspective, I do not see any reason to change the ways in which we are dealing with things. That includes, first of all, that we discuss things openly. And second of all, that we have everything that is in PHP core under the PHP License - apart from bundled libraries, including the Zend engine and TSRM, of course. We btw discussed some time ago that everything in core in fact should be under the PHP License. And so far we do not have a single exception. Instead, we are continuing to replace parts of PHP that have a different license with our own rewrites/re-implementations. If we were to change the way PHP core components are developed or licensed, we would not only scare people away, but would also run the risk of this getting completely out of control.

But when we do not change anything, we do not make any progress. So let's all rethink that.

So we need a better PDO core and better PDO drivers. And that means we need to come up with a better callback infrastructure between main PDO and its various drivers. And that requires some insight in the actual databases, where there are two problems. First, some vendors would only be willing to give us PHP developers insight if we or our companies signed a NDA. And second, the vendors do not usually talk about their internals unless they absolutely have to. And if they do, then this traditionally would be done under a NDA as well. But open source development and NDAs do not really mix. So the NDA way is out.

Now something between completely open and completely closed is (and the Apache Software Foundation does this with quite some success) to use CLAs.

Since I am absolutely against any change of PHP core development because any change might spread out to larger parts of PHP development, let's only deal with PDO here.

That said, PDO main belongs in PHP core, which means that it has to be done under the PHP License which is pretty much incompatible with a CLA. But we can still develop PDO drivers in PECL. And there we can even allow different licenses, at least as long as they are compatible to our OSI approved license. Actually, we already have PECL extensions where people can only contribute after agreeing to some special terms.

At this point we could live happily ever after - if only there weren't the issue of developing a high-performing callback infrastructure. Oh, and we still would need to find people to work on PDO main.

So how do we come up with working specs and people who can contribute?

Right now, we have people contributing to PHP on a regular basis from companies like Google, IBM, MySQL, Oracle, Yahoo, Zend and a bunch of various others - be it large companies, database companies for the PDO matters, established open source companies, or small private companies to freelancers like you and I once were.

Clearly companies can find ways to work on open source without CLAs. And if any company has a problem discussing the protocol of callbacks without having everyone else first sign a CLA, then I can only say that it is their very own problem and potential market disadvantage. Pretty polemic answer, sure. But PHP has gained a lot of momentum in the previous few years and we, the PHP contributors, have accomplished that with a hell lot of effort. Private efforts for the most (and some paid efforts as well). Each and every one of us. So I do not feel the slightest temptation whatsoever, to give even the tiniest piece to any closed company. Plus having to sign a CLA might not be possible to all people working on PDO so far. And with that in mind, I would rather live with having an imperfect callback infrastructure in PDO than losing a list of respected open source contributors.

Sorry to say this, but I do not want a CLA in any part of PHP core.

That of course means that we still have to talk to database vendors. And, guess what, we probably welcome any kind of contribution. Just as we have always gladly accepted any kind of contribution. And maybe the actual problem is starting up a discourse with the vendors. Maybe the problem here is that there is currently no PHP entity. The only thing close to a PHP entity is the so-called PHP Group. But other than having a mail address that goes to some people mentioned on the credits page, the PHP Group does not appear to be doing much of anything. So maybe what we really need to do is fix this situation and recreate the PHP Group. Restart it with people that actually work on PHP and know the internals. Maybe just like PEAR Group by voting; maybe like the Apache Software Foundation; maybe in a completely different way.

Parting words: If you are wondering what PDO 2 is, well so am I. And how is this open source development?

Blogged with Flock

Labels: PHP

Quite some time ago, there was the idea of a DualIterator for PHP's SPL extension.

But right after providing it, I had to learn that allowing two sub Iterators only is too limiting. So when I got asked by Johannes what open TODOs I have left for upcoming PHP 5.3, I thought of it again. He even offered me a PHP implementation of his own. And he did not hesitate to tell me that he would rather see it use SplObjectStorage to store sub Iterators, rather than the Array he used. As he also allowed associative information to be stored along with sub Iterators, I sat down today to add this feature in SplObjectStorage. Having added that, I came up with the below implementation. Now, if anyone is interested, then please help in testing this out. If you do so, please put your tests in phpt format and send me your tests.

<?php
/** @file appenditerator.inc
 * @ingroup SPL
 * @brief class MultipleIterator
 * @author  Johannes Schlueter
 * @author  Marcus Boerger
 * @date    2003 - 2008
 *
 * SPL - Standard PHP Library
 */

/** @ingroup SPL
 * @brief   Iterator that iterates over several iterators one after the other
 * @author  Johannes Schlueter
 * @author  Marcus Boerger
 * @version 1.0
 * @since PHP 5.3
 */
class MultipleIterator implements Iterator {
    /** Inner Iterators */
    private $iterators;

    /** Flags: const MIT_* */
    private $flags;

    /** do not require all sub iterators to be valid in iteration */
    const MIT_NEED_ANY = 0;

    /** require all sub iterators to be valid in iteration */
    const MIT_NEED_ALL  = 1;

    /** keys are created from sub iterators position */
    const MIT_KEYS_NUMERIC  = 0;

    /** keys are created from sub iterators associated infromation */
    const MIT_KEYS_ASSOC  = 2;

    /** Construct a new empty MultipleIterator
     * @param flags MIT_* flags
     */
    public function __construct($flags = self::MIT_NEED_ALL|self::MIT_KEYS_NUMERIC)
    {
        $this->iterators = new SplObjectStorage();
        $this->flags = $flags;
    }

    /** @return current flags MIT_* */
    public function getFlags()
    {
        return $this->flags;
    }

    /** @param $flags new flags. */
    public function setFlags($flags)
    {
        $this->flags = $flags;
    }

    /** @param $iter new Iterator to attach.
     * @param $inf associative info forIteraotr, must be NULL, integer or string
     *
     * @throws IllegalValueException if a inf is none of NULL, integer or string
     * @throws IllegalValueException if a inf is already an associated info
     */
    public function attachIterator(Iterator $iter, $inf = NULL)
    {
        
        if (!is_null($inf))
        {
            if (!is_int($inf) && !is_string($inf))
            {
                throw new IllegalValueException('Inf must be NULL, integer or string');
            }
            foreach($this->iterators as $iter)
            {
                if ($inf == $this->iterators->getInfo())
                {
                    throw new IllegalValueException('Key duplication error');
                }
            }
        }
        $this->iterators->attach($iter, $inf);
    }

    /** @param $iter attached Iterator that should be detached. */
    public function detachIterator(Iterator $iter)
    {
        $this->iterators->detach($iter);
    }

    /** @param $iter Iterator to check
    * @return whether $iter is attached or not
    */
    public function containsIterator(Iterator $iter)
    {
        return $this->iterator->contains($iter);
    }

    /** @return number of attached Iterator instances. */
    public function countIterators()
    {
        return $this->iterators->count();
    }

    /** Rewind all attached Iterator instances. */
    public function rewind()
    {
        foreach($this->iterators as $iter)
        {
            $iter->rewind();
        }
    }

    /**
    * @return whether all or one sub iterator is valid depending on flags.
    * In mode MIT_NEED_ALL we expect all sub iterators to be valid and
    * return flase on the first non valid one. If that flag is not set we
    * return true on the first valid sub iterator found. If no Iterator
    * is attached, we always return false.
    */
    public function valid()
    {
        if (!sizeof($this->iterators)) {
            return false;
        }
        // The following code is an optimized version that executes as few
        // valid() calls as necessary and that only checks the flags once.
        $expect = $this->flags & self::MIT_NEED_ALL ? true : false;
        foreach($this->iterators as $iter)
        {
            if ($expect != $iter->valid())
            {
                return !$expect;
            }
        }
        return $expect;
    }

    /** Move all attached Iterator instances forward. That is invoke
    * their next() method regardless of their state.
    */
    public function next()
    {
        foreach($this->iterators as $iter)
        {
            $iter->next();
        }
    }

    /** @return false if no sub Iterator is attached and an array of
    * all registered Iterator instances current() result.
    * @throws RuntimeException      if mode MIT_NEED_ALL is set and at least one
    *                               attached Iterator is not valid().
    * @throws IllegalValueException if a key is NULL and MIT_KEYS_ASSOC is set.
    */
    public function current()
    {
        if (!sizeof($this->iterators))
        {
            return false;
        }
        $retval = array();
        foreach($this->iterators as $iter)
        {
            if ($it->valid())
            {
                if ($this->flags & self::MIT_KEYS_ASSOC)
                {
                    $key = $this->iterators->getInfo();
                    if (is_null($key))
                    {
                        throw new IllegalValueException('Sub-Iterator is associated with NULL');
                    }
                    $retval[$key] = $iter->current();
                }
                else
                {
                    $retval[] = $iter->current();
                }
            }
            else if ($this->flags & self::MIT_NEED_ALL)
            {
                throw new RuntimeException('Called current() with non valid sub iterator');
            }
            else
            {
                $retval[] = NULL;
            }
        }
        return $retval;
    }

    /** @return false if no sub Iterator is attached and an array of
    * all registered Iterator instances key() result.
    * @throws LogicException if mode MIT_NEED_ALL is set and at least one
    *         attached Iterator is not valid().
    */
    public function key()
    {
        if (!sizeof($this->iterators))
        {
            return false;
        }
        $retval = array();
        foreach($this->iterators as $iter)
        {
            if ($it->valid())
            {
                $retval[] = $iter->key();
            }
            else if ($this->flags & self::MIT_NEED_ALL)
            {
                throw new LogicException('Called key() with non valid sub iterator');
            }
            else
            {
                $retval[] = NULL;
            }
        }
        return $retval;
    }
}




Blogged with Flock

Labels: PHP

After moving to Switzerland, I decided it is time to replace the more than 10-year-old soft snowboard, bindings, and boots set with something new. Even though my brother and I never had any issues whatsoever, the stuff simply got old and worn out. Having spent the weekend before at the 2007 Vancouver PHP conference at Whistler Mountain, where I had an awesome board, I had quite an idea of what I wanted. So I asked friends for a good boarder shop and ended up in the Snowboard Garage. They really had a clue of what they were selling and happened to not only be a Flow dealer, but also have one last Ride Timeless board. Last but not least, I bought pretty damn good Salomon Malamute boots. So Caitlin and I went happily snowboarding. But after 2.5 days my brand new Flow NXT FS bindings got broken already.

Very disappointed, I looked up the designs of the current Flow bindings and found out that they are either cheap plastic crap or have the same design flaw. That is, the high-back consists of three parts. An outer plastic part that holds the closing device. A middle part that is made of aluminum and is connected to the base plate - but to save material is not made high enough. Actually it is just high enough to offer space for two holes through which two screws connect all three parts. Then the inner part is the actual high-back that supports the boot.

Now obviously on the first time I required some support from the bindings (by taking a free ride slope) the whole construction just broke. What happened was that the nut holding the screw just got torn apart. WOW, even I who only spent one semester in mechanical design know this basic rule: the nut must be stronger than the screw. Because one can replace a screw but not the nut. Actually, what the hell, this happened on the third day and I was still carefully driving so to adjust myself to the new equipment? And it is not cheap stuff at all!

O.k., back to the shop and discussing options. Well as said, I looked up the designs and it turns out only the Flow Team binding is able to give some support without having that design flaw. So I had to pay the difference which actually meant that I had to pay the same amount I spent for the other crap in the first place (hey, Flow, I have to work for money, you know?).

After one week, the new bindings finally arrived and we went again, just to notice the next design flaw: the metal strap that connects the high-back is not fully covered by plastic, so it scratches against the plastic strap that holds the boots on the top. WOW! So this binding will eventually last for a year. Idiots. So I just call you Flaw now - and of course no longer suggest your crap to anyone!

Unfortunately there is nothing that can replace a Flaw binding, so I guess I simply
stay with the crap. Why can't the bindings be as they used to be. Just have an awesome quality like the Ride board, the Salomon boots or the also brand new kjus jacket.