Tim Bray: Java is less scalable than PHP

Tim Bray (Director of Web Technologies at Sun Microsystems - I.L.), who - among many other things - co-edited the XML 1.0 and XML namespace definitions, was invited to the International PHP Conference to give a keynote about "How to combine PHP technology with Java based on Enterprise Systems". I had the pleasure to talk with him and I like his spirit. During his keynote, he presented some very interesting comparison between the popular development "frameworks" PHP, Ruby on Rails (RoR, Rails) and Java.

Begin flamewar!

Re: Begin flamewar!

Metrics with PHP, Ruby and Java

Re: Tim Bray: Java is less scalable than PHP

The dangers of arbitrary-ness

Re: Tim Bray: Java is less scalable than PHP

Re: Tim Bray: Java is less scalable than PHP

Pretty shallow analysis by Tim. Very disappointing.

What about:

1) Security. Tell compliance that it's not important.

2) Operations support. Not too many op guys want to fire up an IDE or browse through code to get runtime metrics.

3) Integration. Outside of WS-* and Rest SOA styles, there are other significant integration challenges that neither PHP or Ruby can tackle.

4) Scalability. Didn't buy his scalability arguments. Like the poster said, it would be nice to see some quantitative metrics to support this claim.

It is disturbing that another engineer in the Sun family is downing Java. With the rumours of Open Sourcing, (ie "we're done, you guys deal with it"), the JRuby investments and presentations by internal thought leaders like this, maybe Java(the language, not the VM) is on it's last breath. I hope not.

Hmmm...

Re: Tim Bray: Java is less scalable than PHP

BTW, It is funny that the people who imediately jump to Java's defense have never tried any of the alternatives, like PHP or RoR.....

Re: Tim Bray: Java is less scalable than PHP

Re: Hmmm...

PHP probably is more scalable - I'd agree with that. On a 1-10 scale I'd say java scales as a 7.5 and PHP as a 9 and ruby as a 4. PHP's resource consumption is quite low, and hosting providers have tuned the hell out of it for over many years in large traffic environments. PHP's problem is that it is horribly unmaintainable.

Re: Tim Bray: Java is less scalable than PHP

Having used Ruby and RoR, I would not select Java for the average CRUD web application by choice.

Re: Tim Bray: Java is less scalable than PHP

Personally, coming from being a J2EE champion back in 1997, my opinion now is that Java, J2EE and the Application Server container approach has been over hyped and over sold.

Having used Ruby and RoR, I would not select Java for the average CRUD web application by choice. And my figures for productivity improvements would make these out of context charts look conservative. I would only consider Java for the most complex of EAI tasks, and even then, only because the required integration libraries are likely to be available just for Java.

BTW, It is funny that the people who imediately jump to Java's defense have never tried any of the alternatives, like PHP or RoR. We all need to keep an open mind.

Re: Tim Bray: Java is less scalable than PHP

Which is one of the reasons why I still stand behind my claim that Ruby on Rails will never become mainstream.

it doesn't scale...

Ruby IDE

There's not even a good IDE for ruby.

Radrails is looking good. It is an Eclipse RPC application.

Steve Punte
Test Lens

He never said "Java is less scalable than PHP"

For Web apps, I've given PHP the edge, because I think building scalable PHP is a little easier. By default, PHP gives you a "shared-nothing" (or at least "shared very little") architecture, which means you’re going to scale out pretty well until your database hits the wall. Java is a much richer system and assumes you're smart enough to know whether a shared-nothing architecture is appropriate or not. The effect is, you have to be smarter to get the same kind of scaling out of Java.

Re: He never said "Java is less scalable than PHP"

I think the actual link that should have been used is his blog entry:

http://www.tbray.org/ongoing/When/200x/2006/11/10/Comparing-Frameworks

In it he says:

For Web apps, I've given PHP the edge, because I think building scalable PHP is a little easier. By default, PHP gives you a "shared-nothing" (or at least "shared very little") architecture, which means you’re going to scale out pretty well until your database hits the wall. Java is a much richer system and assumes you're smart enough to know whether a shared-nothing architecture is appropriate or not. The effect is, you have to be smarter to get the same kind of scaling out of Java.

It's not quite the same as the inflamitory statement "Java is less scalable than PHP".

Re: Tim Bray: Java is less scalable than PHP

Having used Ruby and RoR, I would not select Java for the average CRUD web application by choice.

To which Java stack(s) (including toolsets) are you comparing RoR? Java is a language, set of standard libraries, and a virtual machine. Kind of like Ruby. RoR is a framework.

Re: Tim Bray: Java is less scalable than PHP

Well, he was giving a presentation to a PHP conference, what did you expect?

Still, I find his results a bit disappointing, because overall, I enjoy reading his blog and I usually share a lot of his opinions.

Let's start with the Scaling aspect. There are a few big Web sites that use PHP out there, but they are still nowhere near high-traffic Web sites that Java powers. For these, you need a decent amount of clustering and load-balancing technologies, of which PHP offers none.

Having said that, let's keep in mind that Tim is speaking in terms of Web applications only, which stacks the deck in favor of PHP somewhat. Still, web-server based scaling has limits, and when you hit these limits with PHP, you're stuck, while Java will still be able to provide you with more avenues to explore in order to scale your application to the next level.

The Maintainability aspect is also very disingenuous. RoR more maintainable? For whom? I claim: only very good developers. Again, no question that RoR is more elegant, more modern, more sophisticated and, in many ways, generations ahead of what PHP can do. Still, I claim that for 95% of Web developers out there, maintaining a RoR application will be harder than patching a few PHP files. Which is one of the reasons why I still stand behind my claim that Ruby on Rails will never become mainstream.

The "Development Speed" and "Development Tools" graphs look reasonable to me, although they should be considered more qualitative than quantitative.

--
Cedric
TestNG

Re: Tim Bray: Java is less scalable than PHP

Re: Tim Bray: Java is less scalable than PHP

I read your blog entry. And I must admit it left me fuming.

How on earth can we call ourselves a profession when we openly concede that yes Smalltalk and Lisp and even Ruby are great, but are too hard for the average Java joe to comprehend?

RE: it doesn't scale...

Someone better tell Amazon quickly that their apps wont scale for large enterprise sites, they may not be reading this. :)

Re: Tim Bray: Java is less scalable than PHP

Hi Paul,

I read your blog entry. And I must admit it left me fuming.

That makes me sad... It's just a blog post, you know :-)

How on earth can we call ourselves a profession when we openly concede that yes Smalltalk and Lisp and even Ruby are great, but are too hard for the average Java joe to comprehend?

I'm not sure what the problem is with this view. Like in any other industry, there is some kind of bell-curve distribution in complexity in the Java world. These last 30+ years have taught us that most programmers feel reasonably comfortable with languages that follow the syntax pioneered by C and Basic. This is a simple observation, and the success of C++, Java, C# and Visual Basic should be enough to make this obvious to everyone.

Anyway, I'm not sure there is much point in a debate between our respective perceptions of the average level of programmers out there. I'm just observing that a lot of people are turned off by the Smalltalk and Ruby syntaxes, and even for those who feel comfortable with Ruby, the sophistication of Ruby on Rails can be quite intimidating and, that's my personal claim, an obstacle to its widespread adoption.

The simple reality is that the number of Visual Basic and PHP programmers dwarfs by at least an order of magnitude the number of Java and C++ programmers...

--
Cedric

Ruby Syntax

These last 30+ years have taught us that most programmers feel reasonably comfortable with languages that follow the syntax pioneered by C and Basic.

I'm just observing that a lot of people are turned off by the Smalltalk and Ruby syntaxes....

// C / Java / C#:
for (int i=0; i < 10; ++i) {
  print("i is " + i);
}

# Ruby:
for i in (0..10) do
  print("i is " + i)
end

;; Lisp:
(let ((i 0)) 
  (print (concat "i is" i))
  (setq i (+ i 1)))

Re: Tim Bray: Java is less scalable than PHP

I read your blog entry. And I must admit it left me fuming. How on earth can we call ourselves a profession when we openly concede that yes Smalltalk and Lisp and even Ruby are great, but are too hard for the average Java joe to comprehend?

It has been shown time and time again, that a few programmers with the right intellect and skills will always out perform an army of morons.

All the developers I know, are more than capable of comprehending Lisp, Smalltalk, Ruby or anything else you want to throw at them - once they've been shown!

Re: Tim Bray: Java is less scalable than PHP

.... In our industry there is a massive difference between the best and the rest. Apparently studies have shown that the best developers are over 10 times more productive than the average. I know of no other profession that would tolerate a statistic like that.

Re: He never said "Java is less scalable than PHP"

Re: Tim Bray: Java is less scalable than PHP

A bell curve is fine, but it is the standard deviation that counts. In our industry there is a massive difference between the best and the rest. Apparently studies have shown that the best developers are over 10 times more productive than the average. I know of no other profession that would tolerate a statistic like that.

Re: Tim Bray: Java is less scalable than PHP

For the "standard" web app (what ever that is, and obviously standard will very from one person's experience to the next), I've found that RoR is considerably more productive then any combination of the above.

I have a lot of respect for Tim Bray, and I'm sure that the narrative that went with the slides provided a lot more context.

Re: Tim Bray: Java is less scalable than PHP

Also all the people that say that Lisp or Smalltalk are just too weird or too difficult to understand, don't have a problem with them themselves, it is just those other developers who just can't get it :^).

Re: Tim Bray: Java is less scalable than PHP

...and even for those who feel comfortable with Ruby, the sophistication of Ruby on Rails can be quite intimidating and, that's my personal claim, an obstacle to its widespread adoption.

Re: Tim Bray: Java is less scalable than PHP

.... In our industry there is a massive difference between the best and the rest. Apparently studies have shown that the best developers are over 10 times more productive than the average. I know of no other profession that would tolerate a statistic like that.

I bet the best trial attorneys in the US are at least 10 times better than the rest. Those guys that got OJ off aren't available to everyone. :)

Re: Tim Bray: Java is less scalable than PHP

Ruby may become mainstream, but if it does it will be like Visual Basic. Most people who use it will have no clue about the magic going on under the hood. The biggest different will be there will be no protective covering over the magic...which will probably burn people.

Metaprogramming is black magic for wizards. But I find the idea of people using and abusing things of which they have no comprehension very frightening. So, in my opinion, if you don't understand metaprogramming, you don't understand Python well enough to effectively and safely use it.

Re: Tim Bray: Java is less scalable than PHP

.... In our industry there is a massive difference between the best and the rest. Apparently studies have shown that the best developers are over 10 times more productive than the average. I know of no other profession that would tolerate a statistic like that.

I bet the best trial attorneys in the US are at least 10 times better than the rest. Those guys that got OJ off aren't available to everyone. :)

True,

Yes, but they get paid 10 times as much too :^).

How often does this happen in Software?

Paul.

Re: Tim Bray: Java is less scalable than PHP

...and even for those who feel comfortable with Ruby, the sophistication of Ruby on Rails can be quite intimidating and, that's my personal claim, an obstacle to its widespread adoption.

Interesting viewpoint. I've heard the opposite criticism, that Rails is very inviting at first because it hides all the complexity and it's easy to get up and running, but that there's too much magic going on and thus is hard to debug when there's a problem. Like VB without the safety coating as another poster said. I don't agree with this criticism, but it is a common one. I have seen developers who don't know Ruby and try to use Rails have this problem though.

For me, Spring + Hibernate + Java + JSF/Shale/WebWork + JSP + JSTL + EL is very intimidating!

Re: Tim Bray: Java is less scalable than PHP

For the "standard" web app (what ever that is, and obviously standard will very from one person's experience to the next), I've found that RoR is considerably more productive then any combination of the above.

Well, if there is no singe definition of a "standard" web app, then no such claim about productivity is possible (one can't make a claim about something one can't define).

Re: Ruby Syntax

;; Lisp:(let ((i 0)) (print (concat "i is" i)) (setq i (+ i 1)))

What I really said

Re: Ruby Syntax

These last 30+ years have taught us that most programmers feel reasonably comfortable with languages that follow the syntax pioneered by C and Basic.

I'm just observing that a lot of people are turned off by the Smalltalk and Ruby syntaxes....

Actually Ruby syntax is in the same ballpark as C or Basic. Prefix, not postfix notation, block structure, etc. Lisp and Smalltalk however are very different. Arguably one reason Ruby has caught on among programmers who didn't take to Smalltalk and Lisp is the syntax.

Here's a for loop in Java:

// C / Java / C#:
for (int i=0; i < 10; ++i) {
  print("i is " + i);
}

The Ruby version doesn't look that different:

# Ruby:
for i in (0..10) do
  print("i is " + i)
end

But the lisp version is very different:

;; Lisp:
(let ((i 0)) 
  (print (concat "i is" i))
  (setq i (+ i 1)))

(That might not be quite right; my Lisp is rusty.)

Of course when you get into meta-programming and DSL's Ruby can get very magical. But that's not really a syntax issue.

Steve

Re: Tim Bray: Java is less scalable than PHP

IMO this the developers will never get it attitude is propagated by vendors, who want to keep us dumb and subservient so that they can sell us over hyped and over complex products. Promising to solve all our problems, because naturally we are all too dumb to solve our problems ourselves!

Speak for yourself - The majority of developers and customers are tired of playing that game, and are taking their destinies into their own hands. All the developers I know, are more than capable of comprehending Lisp, Smalltalk, Ruby or anything else you want to throw at them - once they've been shown!

Paul.

Re: Tim Bray: Java is less scalable than PHP

For the "standard" web app (what ever that is, and obviously standard will very from one person's experience to the next), I've found that RoR is considerably more productive then any combination of the above.

Well, if there is no singe definition of a "standard" web app, then no such claim about productivity is possible (one can't make a claim about something one can't define).

Not true. I said that my perception is just a single qualitative data point. People use qualitative data all the time as the basis for decision making. Ask politicians and opinion polsters, they'll tell you all about it.

Paul.

Re: Tim Bray: Java is less scalable than PHP

IMO this the developers will never get it attitude is propagated by vendors, who want to keep us dumb and subservient so that they can sell us over hyped and over complex products. Promising to solve all our problems, because naturally we are all too dumb to solve our problems ourselves!

Speak for yourself - The majority of developers and customers are tired of playing that game, and are taking their destinies into their own hands. All the developers I know, are more than capable of comprehending Lisp, Smalltalk, Ruby or anything else you want to throw at them - once they've been shown!

Paul.

I honestly wish I could say the same, but I can't. I have been dealing with other people's code as part of various jobs for 20 years. I have seen code of great quality and elegance, well documented and robust. But, I have seen an equal quantity of code that is extremely poor.

The 'developers won't get it' is not propagated by vendors (why should it be? - they have nothing to gain by it - they could make as much from selling great tools for Smalltalk, LISP or Ruby - in fact many do!). It is the sad result of bitter experience dealing with the consequences of poor quality developers.

All the developers you know may be capable, but all I can say is I can't say that of all the developers I have known over the years.

Re: Tim Bray: Java is less scalable than PHP

For the "standard" web app (what ever that is, and obviously standard will very from one person's experience to the next), I've found that RoR is considerably more productive then any combination of the above.

Well, if there is no singe definition of a "standard" web app, then no such claim about productivity is possible (one can't make a claim about something one can't define).

Not true. I said that my perception is just a single qualitative data point. People use qualitative data all the time as the basis for decision making. Ask politicians and opinion polsters, they'll tell you all about it.

Paul.

I see what you are saying, but I hope that when we are talking about productivity, we are talking about something more than just opinion and politics. I don't think we would be debating this so fiercely if productivity was purely a qualitative matter - poor choices can be expensive.

Re: Ruby Syntax

IMO one of the reasons why Object technology is so poorly understood by many is because Algol like languages (C, C++ etc) lack uniformity. For example they do not uniformily apply message sends to objects as a way of controlling program flow.

For me, I only really understood OO once I started programming in Smalltalk.

Paul.

Discussions, just as usual

Re: Tim Bray: Java is less scalable than PHP

Hi Steve,

I think you are making my point. If the first time sone else gets to review another persons code is when they've got to maintain it, then we must be doing something wrong.

How will the poor programmers you speak of ever get any feedback?

My point wasn't thatb there aren't a load of poor programmers out there, my point was that we need to get better and improving skills across the board.

silly me

Re: silly me

Re: Tim Bray: Java is less scalable than PHP

Hi Steve,

I think you are making my point. If the first time sone else gets to review another persons code is when they've got to maintain it, then we must be doing something wrong.

How will the poor programmers you speak of ever get any feedback?

Unfortunately, I have often seen poor programmers getting feedback from other poor programmers. The quality of pair programming (or other feedback techniques) depends on the quality of the people.

My point wasn't thatb there aren't a load of poor programmers out there, my point was that we need to get better and improving skills across the board.

Well, yes... but until then.... we have to deal with the current situation, not an idea future one where all programmers are watched by skilled mentors.

Re: Tim Bray: Java is less scalable than PHP

Maybe I'm not making myself clear. This is no future scenario. Many people do this today. Spread the knowledge amongst the team, that way you are stronger than your weakest link. I like think that we are as as strong as our strongest link on any given area of expertise.

People did this long before Agile became fashionable. Back in the day it was called a code review.

Paul.

Re: Tim Bray: Java is less scalable than PHP

My point wasn't thatb there aren't a load of poor programmers out there, my point was that we need to get better and improving skills across the board.

Re: Tim Bray: Java is less scalable than PHP

Maybe I'm not making myself clear. This is no future scenario. Many people do this today. Spread the knowledge amongst the team, that way you are stronger than your weakest link. I like think that we are as as strong as our strongest link on any given area of expertise.

People did this long before Agile became fashionable. Back in the day it was called a code review.

Paul.

I know. But my point is that many people don't do this today, or the review is not undertaken by reviewers of sufficient quality or training to understand that the code they are reviewing is distinctly sub-standard. Sometimes the strongest link isn't that strong. Teams, like individuals, vary in quality.

We have to deal with that situation.

Re: Tim Bray: Java is less scalable than PHP

I know. But my point is that many people don't do this today, or the review is not undertaken by reviewers of sufficient quality or training to understand that the code they are reviewing is distinctly sub-standard. Sometimes the strongest link isn't that strong. Teams, like individuals, vary in quality.

We have to deal with that situation.

Hi Steve,

I actually agree. Some teams are better suited to a language like Ruby than others. That's why choice is a good thing, and one size doesn't fit all.

PaUL.

Re: Tim Bray: Java is less scalable than PHP

The guts of your argument is that this is how things are today, so this is how they will necessarily be for an eternity.

You may be happy with the status quoe, but over 70% project failure is nothing to be proud of in my eyes.

A bell curve is fine, but it is the standard deviation that counts.

Apparently studies have shown that the best developers are over 10 times more productive than the average.

Re: RE: it doesn't scale...

Someone better tell Amazon quickly that their apps wont scale for large enterprise sites, they may not be reading this. :)

I think that the Amazon folks may have other things on their minds: [..] masonhq [..]

Re: Hmmm...

PHP probably is more scalable - I'd agree with that. On a 1-10 scale I'd say java scales as a 7.5 and PHP as a 9 and ruby as a 4. PHP's resource consumption is quite low, and hosting providers have tuned the hell out of it for over many years in large traffic environments.

Re: Tim Bray: Java is less scalable than PHP

Let's start with the Scaling aspect. There are a few big Web sites that use PHP out there, but they are still nowhere near high-traffic Web sites that Java powers. For these, you need a decent amount of clustering and load-balancing technologies, of which PHP offers none.

Re: Tim Bray: Java is less scalable than PHP

J2EE is dead. The heavyweight app server container is dead. The state of the art in java is better represented by tools like hibernate, spring for dependency injection, webwork/struts 2 or spring MVC for the webapp.

Re: He never said "Java is less scalable than PHP"

I think the actual link that should have been used is his blog entry:

http://www.tbray.org/ongoing/When/200x/2006/11/10/Comparing-Frameworks

In it he says:

For Web apps, I've given PHP the edge, because I think building scalable PHP is a little easier. By default, PHP gives you a "shared-nothing" (or at least "shared very little") architecture, which means you’re going to scale out pretty well until your database hits the wall. Java is a much richer system and assumes you're smart enough to know whether a shared-nothing architecture is appropriate or not. The effect is, you have to be smarter to get the same kind of scaling out of Java.

It's not quite the same as the inflamitory statement "Java is less scalable than PHP".

Re: Tim Bray: Java is less scalable than PHP

I know. But my point is that many people don't do this today, or the review is not undertaken by reviewers of sufficient quality or training to understand that the code they are reviewing is distinctly sub-standard. Sometimes the strongest link isn't that strong. Teams, like individuals, vary in quality.

We have to deal with that situation.

Hi Steve,

I actually agree. Some teams are better suited to a language like Ruby than others. That's why choice is a good thing, and one size doesn't fit all.

PaUL.

But this is missing the point. Who decides what a team is suited to, or what technologies they can best handle? It takes a good team to understand the issues. And for a very powerful language like Ruby, with its openness, dynamic nature and metaprogramming capabilities, there are many issues to be considered.

Re: Tim Bray: Java is less scalable than PHP

The guts of your argument is that this is how things are today, so this is how they will necessarily be for an eternity.

You may be happy with the status quoe, but over 70% project failure is nothing to be proud of in my eyes.

I'm a very curious to find out how you came to these two conclusions based on reading my blog entry.

I never said things will be like this forever, just that programming languages gain adoption in small increments (C, then C++, then Java/C#, Basic then Visual Basic, then VB6), and Ruby and Smalltalk are too much of a leap for the programming world as it is today.

A bell curve is fine, but it is the standard deviation that counts.

Really? You realize that it can go in either directions, right?

Apparently studies have shown that the best developers are over 10 times more productive than the average.

I keep seeing this factoid thrown here and there, and I still have no idea what it means, if it's true, or if it's a good thing or a bad thing. And I fail to see its relevance to our current discussion, but as I said in my previous post, I really don't think there is much point in having a debate about two very personal views that we have on our industry, which are, by definition, absolutely subjective and therefore, impossible to prove or disprove.

--
Cedric

Re: Tim Bray: Java is less scalable than PHP

Paul,

I know this wasn't directed at me, but I feel compelled to respond.

I read your blog entry. And I must admit it left me fuming. How on earth can we call ourselves a profession when we openly concede that yes Smalltalk and Lisp and even Ruby are great, but are too hard for the average Java joe to comprehend?

I think "hard" is the wrong word. How is this for you:

The average developer is unwilling to comprehend Smalltalk, Lisp, and even Ruby.

If I remember right, Smalltalk was designed so that a child should be able to program it.

Ruby may become mainstream, but if it does it will be like Visual Basic. Most people who use it will have no clue about the magic going on under the hood. The biggest different will be there will be no protective covering over the magic...which will probably burn people.

I'll say I've been trying to learn Lisp for almost a decade now, and I've just finally done it (well, to the point I can actually program in it). It's not that Lisp is hard or doesn't make sense or anything like that. It's that its syntax is offensive to someone who's first "real" language was Pascal (I'm disqualifying Basic and Logo), followed by C, C++, Java, and Ada. It took me six months to finally latch on to Python - mostly because most tutorials started out in an interactive prompt and because of the syntax - and without it I probably would have never tolerated Lisp. Someday I'll get around to Ruby, but last time I tried it drove me nuts. Why? It's not Python, so none of my Python skills apply, and pretending it's Java defeats the purpose of using Ruby. If you're going successfully use a language, you have to understand it's paradigm(s), and that takes patience.

It has been shown time and time again, that a few programmers with the right intellect and skills will always out perform an army of morons.

Don't forget attitude.

All the developers I know, are more than capable of comprehending Lisp, Smalltalk, Ruby or anything else you want to throw at them - once they've been shown!

Define "comprehend." I had to actively apply metaclasses and descriptors in Python before I could see the point of using the language. I don't think the majority of the developers with whom I've worked will ever grasp such matters. If you read what many write about Python metaprogramming, you'd say they don't have to. Metaprogramming is black magic for wizards. But I find the idea of people using and abusing things of which they have no comprehension very frightening. So, in my opinion, if you don't understand metaprogramming, you don't understand Python well enough to effectively and safely use it.

Re: Tim Bray: Java is less scalable than PHP

As for the difficulty in comprehending metaprogramming. In Smalltalk once you get the idea that a class is an object so must also have a class of it's own which is its meta-class, then the rest is plain sailing. Perhaps in Smalltalk this is easier to grasp because all the objects are there tangible and alive in the image and can be inspected at anytime. So you can inspect class and instance objects and see how they relate.

In Python and Ruby it is perhaps more difficult, since these languages have a clear edit then run mode, where in smalltalk there is no distinction, everything is running all the time.

Anyway my point is that metaprogramming is not that dificult and idea, it is just perspective again IMO, and how you come to it.

Paul.

This is why java.net can host PHP but not Java?

Re: Tim Bray: Java is less scalable than PHP

As for the difficulty in comprehending metaprogramming. In Smalltalk once you get the idea that a class is an object so must also have a class of it's own which is its meta-class, then the rest is plain sailing. Perhaps in Smalltalk this is easier to grasp because all the objects are there tangible and alive in the image and can be inspected at anytime. So you can inspect class and instance objects and see how they relate.

In Python and Ruby it is perhaps more difficult, since these languages have a clear edit then run mode, where in smalltalk there is no distinction, everything is running all the time.

Anyway my point is that metaprogramming is not that dificult and idea, it is just perspective again IMO, and how you come to it.

Paul.

The ability to deal with classes (and meta-classes) as objects and understand their existence is, by itself, not metaprogramming, and does not raise the sort of issues that, as I read things, some of us are describing.

Metaprogramming is the ability for programs to write or manipulate themselves or other programs. It is another layer of abstraction, and does not require OOP.

The problem arises when there is layer upon layer of code changing or generating new code, to the point where following what goes on can be difficult. A language like Ruby has a considerable range of metaprogramming features. For example (to use a non-OOP description) functions can be added, removed or replaced at any point in any path through code.

It is not so much that inexperienced developers don't realise that it is possible to do these things. It is that they may be naive about the consequences, or simply not care, and so use clever tricks leading to code that is meaningless or misleading to the casual reader, and even worse, can sabotage other code because of the ability to redefine on-the-fly methods in 'base' classes.

Things are actually the other way around than you describe; languages like Python and Ruby are both more powerful and more dangerous than Smalltalk because there is so little distinction between editing code and running it. Part of the metaprogramming capability means that new source code can be constructed as the program runs.

This is a real issue in these languages that can be of importance even to good developers. There are issues of unpredictability and stability of code which can be very hard to test. There are examples of this happening with Ruby projects involving some of the best Ruby developers.

Re: Tim Bray: Java is less scalable than PHP

Anyway my point is that metaprogramming is not that dificult and idea, it is just perspective again IMO, and how you come to it.

Re: Tim Bray: Java is less scalable than PHP

Use and/or abuse of Metagprogramming is something else. Yes it can be abused - but then again so can many things. Like always the most important ingredient is the programmer, and if you don't trust yourself then don't use it. Use something else, like Java :^)

I think the real issue here is that people should have the choice to use whatever language they feel is well suited to them and their problem.

Re: Tim Bray: Java is less scalable than PHP

Anyway my point is that metaprogramming is not that dificult and idea, it is just perspective again IMO, and how you come to it.

You very well could be right. I don't know. All I know is that moving from normal Java or C++ programming to metaprogramming in a dynamic language is a rather mind-bending experience.

I don't know if it's the concepts or the transition from the static mindset that is difficult.

I'm fairly certain that most developers will not make the leap unless they see significant $$$ on the other side, and the $$$ are drying up where their at.

I'm also fairly certain that a developer without a strong background in static languages is just as stunted as one without a strong background in dynamic ones.

So maybe this is an educational problem, but a large portion of developers aren't Computer Scientists or Computer Engineers. They just picked up programming - so how to you influence their education?

Re: What I really said

Erik Engbrecht, commenting here, got it right: I should have posted the blog piece before the slides leaked out on the Internet. It is a bit irritating that Igor and TSS went with this flame-bait rather than rushing online with their interpretation of one picture, out of context.

Once again, what I really said is here: http://www.tbray.org/ongoing/When/200x/2006/11/10/Comparing-Frameworks

Re: Tim Bray: Java is less scalable than PHP

Hi Steve,

What I was talking about was comprehending metaprogramming. Being able to inspect instance, class and meta-class objects in Smalltalk did aid comprehension for me. For example I never understood what static methods in C++ were all about, until I saw class methods defined in the Metaclass in Smalltalk.

Use and/or abuse of Metagprogramming is something else. Yes it can be abused - but then again so can many things. Like always the most important ingredient is the programmer, and if you don't trust yourself then don't use it. Use something else, like Java :^)

Re: Tim Bray: Java is less scalable than PHP

Hi Steve,

What I was talking about was comprehending metaprogramming. Being able to inspect instance, class and meta-class objects in Smalltalk did aid comprehension for me. For example I never understood what static methods in C++ were all about, until I saw class methods defined in the Metaclass in Smalltalk.

Static methods have nothing to do with metaprogramming. The use of Meta referring to classes is nothing to do with the use of Meta in 'metaprogramming'. You are confusing two entirely different concepts. Understanding one is not relevant to understanding the other.

Use and/or abuse of Metagprogramming is something else. Yes it can be abused - but then again so can many things. Like always the most important ingredient is the programmer, and if you don't trust yourself then don't use it. Use something else, like Java :^)

Let me explain again what I said in a previous post. The metaprogramming capabilities of languages like Ruby have resulted in actual examples of fragile code even when that code has been produced by experienced Ruby developers. Even highly competent people who have trusted themselves to understand the language have had problems. If even the finest 'ingredients' (programmers) can result in something unpalatable, doesn't that suggest a flaw in your viewpoint? If they have occasional problems producing robust code with Ruby, what hope the rest of us?

Metaprogramming is the writing of programs that write or manipulate other programs (or themselves) as their data or that do part of the work that is otherwise done at compile time during runtime. In many cases, this allows programmers to get more done in the same amount of time as they would take to write all the code manually.

Re: Tim Bray: Java is less scalable than PHP

To do this dynamic languages manipulate (change) class and metaclass objects as a way of changing the behaviour of the running program.

Java does the same thing to a lesser degree, but without classes and metaclasses as true objects, you have to rely upon the reflection API and the dynamic proxy API.

As for the safety of using reflection in Java or in say Ruby, and changing code on the fly using say CGLIB in Java then yes it is dangerous, and I wouldn't want to do it myself.

So does that mean we should all stop using Java?

Personally, I avoid the use of reflection in Java, and leave it to the framework and library builders. I take the same phylosophy to Ruby and it's use of reflection to create DSLs. I leave it to the framework (Rails).

Paul.

re: metaprogramming/ruby

Re: Tim Bray: Java is less scalable than PHP

As for the safety of using reflection in Java or in say Ruby, and changing code on the fly using say CGLIB in Java then yes it is dangerous, and I wouldn't want to do it myself.

Re: Tim Bray: Java is less scalable than PHP

Let me explain again what I said in a previous post. The metaprogramming capabilities of languages like Ruby have resulted in actual examples of fragile code even when that code has been produced by experienced Ruby developers. Even highly competent people who have trusted themselves to understand the language have had problems. If even the finest 'ingredients' (programmers) can result in something unpalatable, doesn't that suggest a flaw in your viewpoint? If they have occasional problems producing robust code with Ruby, what hope the rest of us?

Re: Tim Bray: Java is less scalable than PHP

And the non-metaprogamming capabilities of languages like Java have resulted in actual examples of "fragile" code even when that code has been produced by experinced Java developers. Or more generally, even smart people do dumb things sometimes.

Re: Tim Bray: Java is less scalable than PHP

And the non-metaprogamming capabilities of languages like Java have resulted in actual examples of "fragile" code even when that code has been produced by experinced Java developers. Or more generally, even smart people do dumb things sometimes.

Indeed. Because even smart people do dumb things sometimes, it makes sense to try and encourage the wider use of languages that are robust to help reduce the consequences.

Re: Tim Bray: Java is less scalable than PHP

I could be run, but I believe most (all?) frameworks perform bytecode instrumentation at class load time. Once a class is loaded, it stays the way it is. This isn't too far removed from C++ template metaprogramming, where code generation is done at compile time. Both are very different from changing code while the application is running.

Re: Tim Bray: Java is less scalable than PHP

The fact is Java programmers are doing 'magic' or using frameworks with magic every day: Spring AOP, cglib byte manipulation, Hibernate SQL generation and cglib enabled lazy loading, annnotations, EL in JSP's, etc. To keep things 'simple' Java the language doesn't support all that stuff so it gets added through the back door with external libraries. The magic is happening anyway, and it's too hard for the average developer to keep up with the plethora of magic technologies used in one app. That's why Rails is better for the average developer: yes there's magic, but it's mostly all Ruby magic all the way through.

Unless you want to argue that Java's byte code libraries are too tricky for the average developer to use at all so only the clever developers use them. That'd be an ironic position: Ruby is bad for average developers because it's too easy!

Some teams will learn the hard way how not to use a new approach, but in the long run the more productive approach with the higher level of abstraction, combined with best practices on how to use it, will win out.

Re: re: metaprogramming/ruby

I don't think the answer is to limit the flexibility of the language. The answer is to be smart and pragmatic about using the language features. Don't let really freaking smart people go off the deep end.

Re: Tim Bray: Java is less scalable than PHP

The difference is that with Ruby, metaprogramming 'magic' IS encouraged as a standard approach for development.

Re: re: metaprogramming/ruby

I don't think the answer is to limit the flexibility of the language. The answer is to be smart and pragmatic about using the language features. Don't let really freaking smart people go off the deep end.

How?

Re: re: metaprogramming/ruby

I don't think the answer is to limit the flexibility of the language. The answer is to be smart and pragmatic about using the language features. Don't let really freaking smart people go off the deep end.

How?

Same way you don't let your uber-programmers write their own web framework instead of using an standard one... good management keeps an eye out and says no.

Re: re: metaprogramming/ruby

I have seen plenty of examples where management was either not up to judging what programmers were doing, or simply trusted them to get the job done.

I have seen, and had to fix, too many projects where developers did go off the deep end, resulting in code filled with clever tricks that led to cryptic code that was a support nightmare.

Which is why I personally prefer to use, and recommend to others when I get the chance, development languages that aren't as flexible as, say, Ruby or Perl.

I have found it too easy to 'go off the deep end' myself - what I thought at the time was clever coding appeared in hindsight

Re: Hmmm...

PHP probably is more scalable - I'd agree with that. On a 1-10 scale I'd say java scales as a 7.5 and PHP as a 9 and ruby as a 4. PHP's resource consumption is quite low, and hosting providers have tuned the hell out of it for over many years in large traffic environments.

Actually you are not talking about scalability here.

Low ressource usage and scalability are not the same.

Actually it is usually impossible to do both.

My take on Tim Bray and monkeys

Re: Ruby Syntax

The Ruby version doesn't look that different:

# Ruby:
for i in (0..10) do
  print("i is " + i)
end

Re: Ruby Syntax


(1..10).each { |i| puts "i is #{i}" }


Re: My take on Tim Bray and monkeys

I'd love to say I don't know who Tim Bray is, so why should I bother? :)

Alas, I do. Heard him speak once at a Java conference...

Alas, programming is yet not taught as Art and what we get is more like "mass production of programmers". With this picture, any claims and language comparison can be made true - the playing field is often too shallow.

Just my 2 cents

Re: Ruby Syntax


(1..10).each { |i| puts "i is #{i}" }


Of course I meant

(0..10).each { |i| puts "i is #{i}" }

:-)

Erik

Re: My take on Tim Bray and monkeys

I'd love to say I don't know who Tim Bray is, so why should I bother? :)

Alas, I do. Heard him speak once at a Java conference...

Tim is someone of significance - I came across his name years ago as he was heavily involved in the design of XML.

Alas, programming is yet not taught as Art and what we get is more like "mass production of programmers". With this picture, any claims and language comparison can be made true - the playing field is often too shallow.

Just my 2 cents

I am not sure we are getting 'mass production' of programmers - at least then we could be surer of consistent quality. Perhaps software design and engineering needs to be taught something like architecture rather than art - a combination of understanding of materials and good design. The main difference between software engineering and other engineering disciplines is the lack of requirement for qualifications. Not many people would use an architect to design their office unless that architect had formal training and was certified by a recognised institution. At one stage in the 80s, it seemed that a similar process could be established for developers in the UK via the British Computer Society, but it did not seem to catch on, for various reasons.

Object Technology, Hybrid languages and learning

Re: My take on Tim Bray and monkeys

I'd love to say I don't know who Tim Bray is, so why should I bother? :)

Alas, I do. Heard him speak once at a Java conference...

Tim is someone of significance - I came across his name years ago as he was heavily involved in the design of XML.

Alas, programming is yet not taught as Art and what we get is more like "mass production of programmers". With this picture, any claims and language comparison can be made true - the playing field is often too shallow.

Just my 2 cents

I am not sure we are getting 'mass production' of programmers - at least then we could be surer of consistent quality. Perhaps software design and engineering needs to be taught something like architecture rather than art - a combination of understanding of materials and good design. The main difference between software engineering and other engineering disciplines is the lack of requirement for qualifications. Not many people would use an architect to design their office unless that architect had formal training and was certified by a recognised institution. At one stage in the 80s, it seemed that a similar process could be established for developers in the UK via the British Computer Society, but it did not seem to catch on, for various reasons.

The issue of traning for me seems very straight forward. People learn best by example. I see programming as essentially a pratical, pragmatic, intellectual exercise. So refering to it as an art is apt IMO.

People tend to pick up such things by working closely with a master crafts person, learning through example by watching on. They also learn through being allowed to make mistakes and having those mistakes critiqued by their seniors.

In my experience, team leads have been provding such mentoring for other programmers for years, one of the problems though was just because a person had a given skill on their CV, the managment expectation was that they knew it.

So I see a learning culture and open and honesty as paramount.

Software is more an art or craft then a science IMO, and we all need to be continuingly learning.

Having said this the technical side of the job is well understood - the what to do. The when and why to do it, and the when not is a art though, but it can be taught.

If we look to other art forms like journalism, or say fashion design and see how they manage and mentor and brng on talent I think we would find a much more appropriate model for ourselves.

Paul.

Re: re: metaprogramming/ruby

I..I...I...I..I.. doesn't translate to the experiences of other teams.

Php Vs Java Vs RoR

Re: Object Technology, Hybrid languages and learning

The real issue in my opinion, is the poor comprehension by many programmers of fundamental ideas like Object Orientation. The reason for this IMO is that their only exposure to OT has been through hybrid languages. So why hybrid? Well the purpose was to leverage the incumbent technology base (both physical and intellectual) rather then to express the new idea (OT) in a high fidelity way.

Re: Ruby Syntax

Far too long and explicit. Use the full power of Ruby. Redefine 'each'....

Re: Object Technology, Hybrid languages and learning

Anyway, I gotta run, but here is an amusing discussion on OO and FP over at Lambda-the-ultimate.

Re: Ruby Syntax

Far too long and explicit. Use the full power of Ruby. Redefine 'each'....

Sorry, I don't get the irony - not understanding what you aim at. This is as explicit to me (though I'm a beginner in Ruby - but experienced in many other languages) as its equivalents in C, C++ or Java (for example).

Re: Ruby Syntax

Far too long and explicit. Use the full power of Ruby. Redefine 'each'....

Sorry, I don't get the irony - not understanding what you aim at. This is as explicit to me (though I'm a beginner in Ruby - but experienced in many other languages) as its equivalents in C, C++ or Java (for example).

class Range
   def each
      i = first
      while i < last
         puts "i is #{i}"
         i += 1
      end
   end
end

Re: Ruby Syntax

Redefine 'each'...

Ruby is an amazing language - elegant and powerful, but if it is to ever be a serious language for substantial projects, this is one of the matters that will have to be addressed.

Re: Object Technology, Hybrid languages and learning

Hi Paul

The real issue in my opinion, is the poor comprehension by many programmers of fundamental ideas like Object Orientation. The reason for this IMO is that their only exposure to OT has been through hybrid languages. So why hybrid? Well the purpose was to leverage the incumbent technology base (both physical and intellectual) rather then to express the new idea (OT) in a high fidelity way.

I guess by hybrid, you mean a language like C++, and by a pure OO language, you would mean something like Smalltalk. As you probably know, OO is not the only paradigm we have. In fact, I contend that OO means so many different things to different things that it's almost meaningless at this point, except to point out that Smalltalk is the canonical example of a pure OO language.

As you know, OO is not the only paradigm we have. In fact, as concurency becomes a bigger concern, we're seeing the functional paradigm being considered more and more. We're already seeing it coming to fruition in at least one mainstream programming language - C# 3.0 "Why Functional Programming Matter". That paper expouses the virtues of pure functional programming, but most people don't see OO and FP as mutually exclusive. You mentioned Lisp, and as you probably know, the very powerful CLOS was largely built out of Lisp itself, but Lisp is hardly a pure OO language, and Lisp programmers can choose the appropriate tools - macros, CLOS, functional, for the domain they are writing to.

Anyway, I gotta run, but here is an amusing discussion on OO and FP over at Lambda-the-ultimate.

Re: Ruby Syntax

Ruby is an amazing language - elegant and powerful, but if it is to ever be a serious language for substantial projects, this is one of the matters that will have to be addressed.

Re: Ruby Syntax

It depends on the team. This is why I keep on mentioning choice. I can't see why you have a difficulty with this concept. People that know what they are doing should be able to choose to use expressive languages if they want to. Why dumb down everyone?

If Ruby doesn't work for you don't use it - but don't try and enforce your choices on everyone else.

Re: Ruby Syntax

But you have over-written a method in a core class. The results are likely to be disastrous. The point is, any section of Ruby code anywhere, written by anyone, can do this. Or, they can overwrite any method in your code.

This sort of thing has been seen before. It used to happen frequently in Smalltalk (although Smalltalk is less fragile, as such method name conflicts can usually be detected at the point of compilation; Ruby has no compilation).
.

Re: Object Technology, Hybrid languages and learning

I agree. The problem I'm highlighting is that OT has been diluted, confused and banded around as amarketing term for so long that it has become meaningless.

I would say that 90% of Java programmers I come across haven't a full grasp of OT and happily write procedural code unknowingly.

Re: Ruby Syntax

It is not a problem, because people just don't do it.

Re: Ruby Syntax

Though I agree that Steve's point of view (at least in the way he expresses it here) seems exagerated we must also have a thought about what can happen if 'people ever do it' (either by inadvertance or malignantly): what will it cost to find the cause of the problem, what can break in our system, does it open the possibility for security breaches etc.

Re: Ruby Syntax

Given the infinite permutations that any non-trivial program can be expressed as, Java isn't going to save you either. Some people seem to want the language designers to impose the strait jacket, while I'd rather have the team guidelines impose it. There's many valid reasons for the "safe choice", but it comes at a cost, and should be weighed as such by the myriad of environmental factors that a team has.

Re: Ruby Syntax

So what is there to stop people overriding 'clone' or 'equals' or 'hashCode' in Java giving them new inappropriate semantics? Or implementing/overriding Comparable with inappropriate semantics? Bang goes your use of collections and comparators.

BTW, this happening frequently with Smalltalk? Where? Have you evidence? I have used Smalltalk since 1993 and I have seen no evidence of what you describe as a wide spread practice.

I just can't understand you. Rather than all this FUD, why not just tell people not to do certain things?

I just think you are creating a false argument. People have been using Ruby since 1993, I've been using it since 2005 and I have not come across a single instance of what you describe.

Tim Bray is [in]famous

I'd love to say I don't know who Tim Bray is, so why should I bother? :)

Tim is someone of significance - I came across his name years ago as he was heavily involved in the design of XML.

Re: Object Technology, Hybrid languages and learning

He still refuses to say what is a robust language and what isn't. I guess he got caught off guard on someone picking that up.

Re: Tim Bray is [in]famous

I'd love to say I don't know who Tim Bray is, so why should I bother? :)

Tim is someone of significance - I came across his name years ago as he was heavily involved in the design of XML.

Yes. He is responsible for XML.

Does that merit a medal of honor, or a firing squad? ;-)

Peace,

Cameron Purdy
Tangosol Coherence: The Java Clustered Cache

Re: Ruby Syntax

Though I agree that Steve's point of view (at least in the way he expresses it here) seems exagerated...

Assembly for scalability!

Re: Object Technology, Hybrid languages and learning

He still refuses to say what is a robust language and what isn't. I guess he got caught off guard on someone picking that up.

Not at all.

It is just that I know exactly what will happen. I will say 'Java' is robust (although I would also definitely put modern Smalltalks in that category), then you will talk about being able to do harm in any language, I will talk about how all that matters is relative safety, then it will go downhill from there.

Sorry if this seemed impolite, but I believe we have such diametrically opposed opinions, that I can't see any point in debating, as we won't get anywhere.