staying neutral

Having been at a Conference a few days ago, I was reminded of something. I have several times been asked how companies can help PHP. And my typical answer is about contributing tests and making sure PHP runs under or with their products, be that operating systems, databases or other back ends; or tools in any way. However I just updated the PHP Version info in a few hundred source files from 'PHP Version 5' to 'PHP Versoin 6'. And I also found that the License file lacked an update to 2009. And at this point you might already guess what the addition to my typical answer is. Why can Zend not simply change the license of the Zend Engine to PHP License?

Why do we want this?

Becasue it creates issues with using PHP. And we do not even inform people about it, because we are silent about this fact.

So is there a reason why this has not happened already long ago?

Some people claim it is because of ownership. So one question actually is who owns the code. And being a friend of numbers and cryptic shell code, here is some data:

$> cvs annotate Zend 2>/dev/null |
   grep -v '(changelo ' |    awk '/[0-9.]*.*\(/ {
     L++
     C[substr($2,2)]++
   }
   END {
     for(n in C) {
       printf "%5d %4.1f%% %s\n", C[n], C[n] * 100 / L, n
     }
   }' |
   sort -r

36863 25.6% dmitry
29428 20.4% andi
15323 10.6% tony2001
 9541  6.6% stas
 6894  4.8% andrei
 6665  4.6% zeev
 6504  4.5% felipe
 5711  4.0% helly
 5381  3.7% scottmac
 5341  3.7% sebastia
 2760  1.9% iliaa
 2607  1.8% nlopess
 1731  1.2% derick
 1550  1.1% sniper
 1108  0.8% kraghuba

I skipped changelog and everyone who contributed less than 1000 lines got stripped manually (which makes it 15 lines).

Does that mean Zend owns the engine? In terms of the License they do. In terms of contribution I do not. Simply because other people like me contributed does not mean we share ownership. Not even if there wasn't a single line written by Zend employees left. So is the other way round true, and Zend owns the engine because employees have written the engine? Also not, because we do not know whether they did it in their spare time or were paid for it. So why does Zend own the engine? Simply put, the License file says so.

And there is another area that uses an arbitrary License. The TSRM library. Here is the data (again top 15 lines only):

957 21.5% andi
804 18.0% zeev
785 17.6% dmitry
402  9.0% dbeu
384  8.6% sascha
242  5.4% sebastia
167  3.7% rvenkat
136  3.1% tony2001
 99  2.2% wez
 65  1.5% dreid
 63  1.4% iliaa
 53  1.2% stig
 53  1.2% helly
 50  1.1% sniper
 28  0.6% sesser

The fun is that hardly anyone but Windows users need this library. Yet it interferes with using PHP in the same way the Zend License does. Simply because it is an arbitrary License.

Blogged with the Flock Browser

Labels: PHP

# posted by helly @ 02:03 2 Comments Links to this post

Unfortunately this year I could not go snowboarding before PHP Quebec 2009 Conference because of my work schedule. So I flew from NYC to Montreal only on Tuesday afternoon. The conference yet again changed the Hotel and was this year held in the Hilton Montreal Bonaventure. Personally I liked the Sofitel better as a hotel, but for the conference the new hotel is definitively a better choice. We had more rooms, generally more space and - a working WIFI. Probably the first conference with a working WIFI. Thanks Anna, Sylvain and Yann. The conference itself was good as always and we had a lot of fun during the evenings - of course. I gave two planned talks and I had to jump in for my dear friend Sara, who could not come herself because she got sick right before the conference. Luckily my friend Johannes agreed to join me for the jump in on the extension writing talk. And we really both enjoyed the Worst PHP Practice talk, for which actually my wife Caitlin had the idea - thanks for that (and many other things). The talks are online:

• Objects for the Masses
• PHP Extension Writing
  
• Worst PHP Practice

Other talks were luckily not too much about PHP 5.3. Instead it turns out that people are interested in the current stable platform PHP 5.2 and are willing to get us core developers more time to get the new features, most notably closures , namespaces and Phar done right (to name a few I contributed to). If you are still interested in more about PHP 5.3, check out Johannes' talk . Something else of interest; I was happy to hear and see that other presenters used Google Chart API in their talks for examples.

At the last night of the conference we went to Le Deux Pierrots. And of course we finished the conference week with a visit to Sucrerie de la Montagne. And believe it or not, we poured sugar syrup onto soup and coffee as well as over mashed potatoes, meet - and - sugar pie. Finally in the afternoon I had to leave to the airport - learning from last years stress with getting the plane, this year I booked a later flight.

Labels: PHP

Finally.

After years on working on RE2C to make it stable and ready to replace flex-based scanners, work on replacing PHP's flex-based scanner with a RE2C-based one has begun. Actually the work was started by Nuno Lopes and Scott MacVicar on 14th of February 2008, after I happily informed them that I considered RE2C ready for this task. I joined the work a day later, contributed a few ideas and helped with minor RE2C improvements. Today on the 15th of February we can already run make test and get over 98% PASS.

This encouraged me to spend a bit more time on RE2C and release 0.13.2, which you can download here (sorry but it is on sourceforge).

Thanks to Scott, we also have a subversion and Trac setup here, where you can follow our development. If you want to experiment yourself, or even help in our efforts, then you can checkout from svn://whisky.macvicar.net/php-re2c.

Blogged with Flock

Labels: PHP, RE2C

So I've had enough of different PHP versions compiled in different modes (ZTS or DEGBUG) and having to change the php.ini file every now and then on my dev machines. So I added conditional INI support:


I used square brackets for the conditionals, as it seemed the way that comes with the least potential for INI handling. This is because right now we ignore section names anyway.

The patch can be downloaded here and allows for the following:

• String comparison using operators === and !==.
• All other operators convert left and right operands to integers first, just as INI handling would do for value evaluation.
• Support for operators <, <=, >, >=, ==, !=, && and ||.
• Braces can be used to force an evaluation order.
  
• No support for unary operator ! or ternary operator ?:.
• No support for functions.
• No direct boolean evaluation, that is operators are required.
• Conditions can be nested.
  

If you are interested in this stuff, you can either comment on this blog entry or follow the discussion on internals@php.net here.

Blogged with Flock

Labels: PHP

So after the initial uproar on last week's attempts to put parts of PHP development under the terms of a CLA (read proposal here), a bunch of us actually spent some time in finding solutions for one way or the other. I don't want to bother you with more details on the why.

But, one thing for certain, we want PDO.

As the reasoning, this was discussed enough, so I'll jump directly to my ideas for a solution.

• Develop a PECL CLA that can optionally be used for PECL projects.
• If necessary, adapt the PHP License, so that it works nicely together with the CLA.
  
• The projects that want a CLA can choose between the PHP License or LGPL.
• Change the PECL web site so that projects can opt-in to using the CLA.
• Arrange it so that projects cannot drop the CLA flag.
  
• Add a user/CLA/project table to the PHP user database, and use this in CVS ACLs.
  
• Create a new CVS module php-default.
• Move all extensions that can be disabled and are not required for others to PECL.
• Link everything under php-src plus a default selection of extensions to php-default.
• Let us once and for all ban CLAs from php-src aka PHP core.
  
• Start developing PDO as part of CVS module php-src.

Sorry for not writing this earlier. So how does this idea sound? Check out answers to the mail here.

The Borg Queen to Kathryn Janeway: 'Do as all pragmatists do, compromise'. And it is no secret that I am a pragmatist.

Blogged with Flock

Labels: PHP

These are my very private thoughts on what is going on with PDO.

First of all, having decent database support is becoming more and more important to PHP because there is hardly any PHP application left that does not have some kind of database backend. And for that exact reason, a few people started PDO some time ago.

Technically speaking, right now we are on the second PDO version - well starting to count unix-like from version 0.

The initial PDO developed by Sterling Hughes and me never made it into PHP. It was, however, the base for the initial version we later put into PHP, as it proved that our main design goals worked: unified API through a base class with different database back-ends connected through drivers, where drivers are separate extensions that sit on top of the main PDO extension. After some discussions at conferences - such as the International PHP Conference and LinuxTag - it was Wez Furlong who came up with the initial implementation of PDO version 1. It had a nicely worked-out callback infrastructure that could support a bunch of database backends. And, indeed, we got a few of those in short time.

So what we have right now is a more or less working PDO that suffers from the following:

• Some databases have suboptimal drivers (often slower than native extensions).


• No support from database vendors to help improve drivers (for various reasons).


• A maintainance nightmare, where PDO in different PHP branches is completely out of synch.


• Limited functionality in PDO (like missing metadata support).


• Missing developers for the main PDO extension.

So the main questions are: how do we attract more developers to PDO, and how do we improve PDO? And all this to boot without scaring away current developers.

Now thanks to Wez for the PDO 1 specification effort and talking to a bunch of database vendors, trying to integrate them into PDO development. Because only with a good documentation can we make any progress whatsoever, be it a full rewrite or just continuous development. Btw, my thanks also go to the countless people that wrote the PDO documentation.

However, I completely disagree with the way in which PDO 2 was started. The main reason is that throughout the preparation phase not a single developer had any clue whatsoever of what was going on - not even that something was going on at all! If that is not the case, then obviously those who did know were under some kind of NDA. It actually appears that there is a PDO development mailing list, which I was not aware of until two days ago (for PDO 0 we used my private mail server). And even if I had been aware of it, it still would not have been the PHP way of doing things - where the PHP way is doing development openly, or at least openly after a short startup time. Either way, to most people who read the announcement of a CLA in the PHP ACLs some time ago and the PDO 2 development proposal recently, this is a clear indication that the upcoming PDO version won't be open source. Because open means open and the PHP community in particular has always been especially open. Even though, as we recently discussed again, we might have a reason to slow down traffic on internals@, in general we all profited from our openness.

From my very own perspective, I do not see any reason to change the ways in which we are dealing with things. That includes, first of all, that we discuss things openly. And second of all, that we have everything that is in PHP core under the PHP License - apart from bundled libraries, including the Zend engine and TSRM, of course. We btw discussed some time ago that everything in core in fact should be under the PHP License. And so far we do not have a single exception. Instead, we are continuing to replace parts of PHP that have a different license with our own rewrites/re-implementations. If we were to change the way PHP core components are developed or licensed, we would not only scare people away, but would also run the risk of this getting completely out of control.

But when we do not change anything, we do not make any progress. So let's all rethink that.

So we need a better PDO core and better PDO drivers. And that means we need to come up with a better callback infrastructure between main PDO and its various drivers. And that requires some insight in the actual databases, where there are two problems. First, some vendors would only be willing to give us PHP developers insight if we or our companies signed a NDA. And second, the vendors do not usually talk about their internals unless they absolutely have to. And if they do, then this traditionally would be done under a NDA as well. But open source development and NDAs do not really mix. So the NDA way is out.

Now something between completely open and completely closed is (and the Apache Software Foundation does this with quite some success) to use CLAs.

Since I am absolutely against any change of PHP core development because any change might spread out to larger parts of PHP development, let's only deal with PDO here.

That said, PDO main belongs in PHP core, which means that it has to be done under the PHP License which is pretty much incompatible with a CLA. But we can still develop PDO drivers in PECL. And there we can even allow different licenses, at least as long as they are compatible to our OSI approved license. Actually, we already have PECL extensions where people can only contribute after agreeing to some special terms.

At this point we could live happily ever after - if only there weren't the issue of developing a high-performing callback infrastructure. Oh, and we still would need to find people to work on PDO main.

So how do we come up with working specs and people who can contribute?

Right now, we have people contributing to PHP on a regular basis from companies like Google, IBM, MySQL, Oracle, Yahoo, Zend and a bunch of various others - be it large companies, database companies for the PDO matters, established open source companies, or small private companies to freelancers like you and I once were.

Clearly companies can find ways to work on open source without CLAs. And if any company has a problem discussing the protocol of callbacks without having everyone else first sign a CLA, then I can only say that it is their very own problem and potential market disadvantage. Pretty polemic answer, sure. But PHP has gained a lot of momentum in the previous few years and we, the PHP contributors, have accomplished that with a hell lot of effort. Private efforts for the most (and some paid efforts as well). Each and every one of us. So I do not feel the slightest temptation whatsoever, to give even the tiniest piece to any closed company. Plus having to sign a CLA might not be possible to all people working on PDO so far. And with that in mind, I would rather live with having an imperfect callback infrastructure in PDO than losing a list of respected open source contributors.

Sorry to say this, but I do not want a CLA in any part of PHP core.

That of course means that we still have to talk to database vendors. And, guess what, we probably welcome any kind of contribution. Just as we have always gladly accepted any kind of contribution. And maybe the actual problem is starting up a discourse with the vendors. Maybe the problem here is that there is currently no PHP entity. The only thing close to a PHP entity is the so-called PHP Group. But other than having a mail address that goes to some people mentioned on the credits page, the PHP Group does not appear to be doing much of anything. So maybe what we really need to do is fix this situation and recreate the PHP Group. Restart it with people that actually work on PHP and know the internals. Maybe just like PEAR Group by voting; maybe like the Apache Software Foundation; maybe in a completely different way.

Parting words: If you are wondering what PDO 2 is, well so am I. And how is this open source development?

Blogged with Flock

Labels: PHP

Quite some time ago, there was the idea of a DualIterator for PHP's SPL extension.

But right after providing it, I had to learn that allowing two sub Iterators only is too limiting. So when I got asked by Johannes what open TODOs I have left for upcoming PHP 5.3, I thought of it again. He even offered me a PHP implementation of his own. And he did not hesitate to tell me that he would rather see it use SplObjectStorage to store sub Iterators, rather than the Array he used. As he also allowed associative information to be stored along with sub Iterators, I sat down today to add this feature in SplObjectStorage. Having added that, I came up with the below implementation. Now, if anyone is interested, then please help in testing this out. If you do so, please put your tests in phpt format and send me your tests.

<?php
/** @file appenditerator.inc
 * @ingroup SPL
 * @brief class MultipleIterator
 * @author  Johannes Schlueter
 * @author  Marcus Boerger
 * @date    2003 - 2008
 *
 * SPL - Standard PHP Library
 */

/** @ingroup SPL
 * @brief   Iterator that iterates over several iterators one after the other
 * @author  Johannes Schlueter
 * @author  Marcus Boerger
 * @version 1.0
 * @since PHP 5.3
 */
class MultipleIterator implements Iterator {
    /** Inner Iterators */
    private $iterators;

    /** Flags: const MIT_* */
    private $flags;

    /** do not require all sub iterators to be valid in iteration */
    const MIT_NEED_ANY = 0;

    /** require all sub iterators to be valid in iteration */
    const MIT_NEED_ALL  = 1;

    /** keys are created from sub iterators position */
    const MIT_KEYS_NUMERIC  = 0;

    /** keys are created from sub iterators associated infromation */
    const MIT_KEYS_ASSOC  = 2;

    /** Construct a new empty MultipleIterator
     * @param flags MIT_* flags
     */
    public function __construct($flags = self::MIT_NEED_ALL|self::MIT_KEYS_NUMERIC)
    {
        $this->iterators = new SplObjectStorage();
        $this->flags = $flags;
    }

    /** @return current flags MIT_* */
    public function getFlags()
    {
        return $this->flags;
    }

    /** @param $flags new flags. */
    public function setFlags($flags)
    {
        $this->flags = $flags;
    }

    /** @param $iter new Iterator to attach.
     * @param $inf associative info forIteraotr, must be NULL, integer or string
     *
     * @throws IllegalValueException if a inf is none of NULL, integer or string
     * @throws IllegalValueException if a inf is already an associated info
     */
    public function attachIterator(Iterator $iter, $inf = NULL)
    {
        
        if (!is_null($inf))
        {
            if (!is_int($inf) && !is_string($inf))
            {
                throw new IllegalValueException('Inf must be NULL, integer or string');
            }
            foreach($this->iterators as $iter)
            {
                if ($inf == $this->iterators->getInfo())
                {
                    throw new IllegalValueException('Key duplication error');
                }
            }
        }
        $this->iterators->attach($iter, $inf);
    }

    /** @param $iter attached Iterator that should be detached. */
    public function detachIterator(Iterator $iter)
    {
        $this->iterators->detach($iter);
    }

    /** @param $iter Iterator to check
    * @return whether $iter is attached or not
    */
    public function containsIterator(Iterator $iter)
    {
        return $this->iterator->contains($iter);
    }

    /** @return number of attached Iterator instances. */
    public function countIterators()
    {
        return $this->iterators->count();
    }

    /** Rewind all attached Iterator instances. */
    public function rewind()
    {
        foreach($this->iterators as $iter)
        {
            $iter->rewind();
        }
    }

    /**
    * @return whether all or one sub iterator is valid depending on flags.
    * In mode MIT_NEED_ALL we expect all sub iterators to be valid and
    * return flase on the first non valid one. If that flag is not set we
    * return true on the first valid sub iterator found. If no Iterator
    * is attached, we always return false.
    */
    public function valid()
    {
        if (!sizeof($this->iterators)) {
            return false;
        }
        // The following code is an optimized version that executes as few
        // valid() calls as necessary and that only checks the flags once.
        $expect = $this->flags & self::MIT_NEED_ALL ? true : false;
        foreach($this->iterators as $iter)
        {
            if ($expect != $iter->valid())
            {
                return !$expect;
            }
        }
        return $expect;
    }

    /** Move all attached Iterator instances forward. That is invoke
    * their next() method regardless of their state.
    */
    public function next()
    {
        foreach($this->iterators as $iter)
        {
            $iter->next();
        }
    }

    /** @return false if no sub Iterator is attached and an array of
    * all registered Iterator instances current() result.
    * @throws RuntimeException      if mode MIT_NEED_ALL is set and at least one
    *                               attached Iterator is not valid().
    * @throws IllegalValueException if a key is NULL and MIT_KEYS_ASSOC is set.
    */
    public function current()
    {
        if (!sizeof($this->iterators))
        {
            return false;
        }
        $retval = array();
        foreach($this->iterators as $iter)
        {
            if ($it->valid())
            {
                if ($this->flags & self::MIT_KEYS_ASSOC)
                {
                    $key = $this->iterators->getInfo();
                    if (is_null($key))
                    {
                        throw new IllegalValueException('Sub-Iterator is associated with NULL');
                    }
                    $retval[$key] = $iter->current();
                }
                else
                {
                    $retval[] = $iter->current();
                }
            }
            else if ($this->flags & self::MIT_NEED_ALL)
            {
                throw new RuntimeException('Called current() with non valid sub iterator');
            }
            else
            {
                $retval[] = NULL;
            }
        }
        return $retval;
    }

    /** @return false if no sub Iterator is attached and an array of
    * all registered Iterator instances key() result.
    * @throws LogicException if mode MIT_NEED_ALL is set and at least one
    *         attached Iterator is not valid().
    */
    public function key()
    {
        if (!sizeof($this->iterators))
        {
            return false;
        }
        $retval = array();
        foreach($this->iterators as $iter)
        {
            if ($it->valid())
            {
                $retval[] = $iter->key();
            }
            else if ($this->flags & self::MIT_NEED_ALL)
            {
                throw new LogicException('Called key() with non valid sub iterator');
            }
            else
            {
                $retval[] = NULL;
            }
        }
        return $retval;
    }
}




Blogged with Flock

Labels: PHP