Migration to Java 5 at walmart.com

Discussions

News: Migration to Java 5 at walmart.com

  1. Migration to Java 5 at walmart.com (61 messages)

    How do you upgrade one of the busiest web applications of the whole Internet to Java 1.5? This is a challenge under the best conditions, considering that the application must handle up to 7 million sessions and 106 million page views per day. This article is a case study of the Java 1.5 upgrade for the application and subsystems that make up the walmart.com web site.

    Read "Living on the Edge: Migration to Java 5 at walmart.com"

    Threaded Messages (61)

  2. Migration to Java 5 at walmart.com[ Go to top ]

    This is great! It proves, yet again, that Java is working well (and fast enough) in the real world out there. A slap in the face for those snide remarks about Java technology...

    I am yet to see any major .NET endorsements (case studies, articles whatever) that is not a marketing strategy sponsored by Microsoft. Are there any out there that I am not familiar with... ?
  3. Rocky[ Go to top ]

    The author is also a boxer (http://www.eugeneciurana.com/media/BoxingLQ.mpg), which is definitely offtopic, but still worthwile :)
  4. The Leviathan[ Go to top ]

    Definitely off-topic but interesting story on the bowels of the Wal*Mart Leviathan.
  5. Apache and Tomcat ??[ Go to top ]

    Is that correct? The wall mart site is running with Apache and Tomcat. No JBoss? No Websphere?
    Curious.
  6. Apache and Tomcat ??[ Go to top ]

    Apache and Tomcat, that's it.

    Think about why for a bit.

    How well can you optimize every aspect of Websphere? Hrm... something to think about...

    Cheers,

    E
  7. We're sorry![ Go to top ]

    We're sorry!



    The Walmart.com Web site is temporarily closed. Please visit again soon.

    In the meantime, if you need any assistance, please email us anytime at Help at Walmart dot com or call our Customer Service during the hours of 6 a.m. to 1 a.m. CST at 1-800-966-6546.

    Thank you.
  8. We're sorry![ Go to top ]

    I guess some people have never heard of maintenance windows...

    Cheers!
  9. We're sorry![ Go to top ]

    Maintenance windows. Lots of content to refresh daily.

    Cheers!
  10. Apache and Tomcat ??[ Go to top ]

    What version of Tomcat are you using?
  11. Apache and Tomcat ??[ Go to top ]

    Thanks for the reply.
    I have used only Tomcat in previous projects and have had to debate if Tomcat was scalable and ready for production use. Some articles on the web say it is good for development only but I disagree.
    The use of Tomcat and Apache at Wallmart serving 100 million page hits per day proves that Tomcat is truly production ready.
    What version of Tomcat are you using?
    Thanks for the info.
    Graeme
  12. good breakdown...[ Go to top ]

    ... for anyone making the migration. 8 week cycles seems reasonable as well, as there's probably no "urgency" behind the move.

    However, there is no way in hell I'd ever work for Walmart... even if I was the Tech Lead.
    You'd basically be the captain of the "cool new ship" that is enslaving millions of inhabitants of third world countries for cheap labor.

    "I just drove the train that took people to the concentration camps... I didn't kill em' !"
  13. good breakdown...[ Go to top ]

    Oh god please shuuuuuuuuuuuuuuuuut up! God forbid they try and make any money. Does Wal-Mart hold a gun to people's heads and force them to work there? No. They CHOOSE to work there. Admit it, you're just against capitalism.

    Can you say 4B / month in sales? Why aren't you attacking Sears, Nordstroms, or Microsoft?
  14. good breakdown...[ Go to top ]

    Oh god please shuuuuuuuuuuuuuuuuut up! God forbid they try and make any money. Does Wal-Mart hold a gun to people's heads and force them to work there? No. They CHOOSE to work there. Admit it, you're just against capitalism.Can you say 4B / month in sales? Why aren't you attacking Sears, Nordstroms, or Microsoft?

    Yeah, I'm still trying to understand why everytime a discussion comes up about some large company that is very successful, they are labeled greedy and I love this one "powerful money machines" as if that's a bad thing.
  15. good breakdown...[ Go to top ]

    ... for anyone making the migration. 8 week cycles seems reasonable as well, as there's probably no "urgency" behind the move.However, there is no way in hell I'd ever work for Walmart... even if I was the Tech Lead.You'd basically be the captain of the "cool new ship" that is enslaving millions of inhabitants of third world countries for cheap labor."I just drove the train that took people to the concentration camps... I didn't kill em' !"

    How does a discussion about Walmart moving to Java, swerve into slave labor and Walmart? I really hate to comment, but how are you enslaving people that otherwise would not have a job?
  16. good breakdown...[ Go to top ]

    "I just drove the train that took people to the concentration camps... I didn't kill em' !"

    Wow, how fucked up do you have to be to bring up the Holocaust in a comparison?

    Dylan it's a Java website - keep it that way.
  17. good breakdown...[ Go to top ]

    "I just drove the train that took people to the concentration camps... I didn't kill em' !"
    Wow, how fucked up do you have to be to bring up the Holocaust in a comparison? Dylan it's a Java website - keep it that way.

    I actually did post more threads about the 1.4 - 5 Migration than I did my shameless "@#$% Walmart" plug.
    The Holocaust / Walmart comparison relates to players like Adolf Eichmann (the main "coordinator" of the Holocaust), and yes, the amount of people effected by such "coordination of business".

    - The End ;)
  18. good breakdown...[ Go to top ]

    Dylan it's a Java website - keep it that way.

    I don't agree. I think it is important to understand how the companies we work for affect the world around us. You can choose to ignore that fact but then may be faced with the lifetime of guilt that some scientists working on the Manhattan project felt after it was first lit off.

    Besides, just Java is boring.


    That said:
    Using the Holocaust to compare against anything but genocide is irresponsible and weakens our ability to express "real" atrocities. I do agree that Dylan should be more precise and thoughtful.

    To the Walmart apologist, Mike Jones:
    Evidence of Walmart's detriment to the US worker, US taxpayer, and society at large is pretty well documented. Anybody with two fingers and access to google can find out as much about it as they want.

    To be honest I fear big corporations like I fear big government. Just as a mob of people has a mentality of its own, large corporations become like a living animal (a monster if you will) where no one group of people is in charge, or responsible for the actions of the corporation at large.

    If I have to take sides I'll go with the humans.
  19. good breakdown...[ Go to top ]

    Dylan it's a Java website - keep it that way.
    I don't agree. I think it is important to understand how the companies we work for affect the world around us. You can choose to ignore that fact but then may be faced with the lifetime of guilt that some scientists working on the Manhattan project felt after it was first lit off.Besides, just Java is boring.

    Frist off he doesn't WORK for Walmart. Second anyone who isn't living under a rock has either read a story or two on Walmart or has seen a documentary on it. Third, paying people ,who would otherwise not have ANY work, a low wage is so far from a Holocaust that it borders on being PATHETIC. So pathetic that even /. people would shudder.

    Look I'm not reading theserverside.com to hear about people's thoughts on abortion, politics, etc. As for fun, yea reading about the Holocaust is BIGTIME fun.

    And I'm not reading this site for fun. If you want fun then read Get Fuzzy, http://www.comics.com/comics/getfuzzy/.
  20. good breakdown...[ Go to top ]

    George... thank your for articulating my comment, as that's exactly what I needed to throw my $0.02 about.
    Look I'm not reading theserverside.com to hear about people's thoughts on abortion, politics, etc
    Brian... I totally agree, but had to throw a little somethin-somethin out there for the people that may be living under a rock.
    So pathetic that even /. people would shudder
    No way, I would have probably got a (Score:5, Insightful) for that one ;)
  21. good breakdown...[ Go to top ]

    http://walmartwatch.com/
  22. good breakdown...[ Go to top ]

    Third, paying people ,who would otherwise not have ANY work, a low wage is so far from a Holocaust that it borders on being PATHETIC.

    I find it quite strange to assume that these people would otherwise not have "any work". I don't happen to live in the US, but afaik, the policy of walmart to import textiles from China has ultimately pushed all its competitors to do exactly the same. So we now have former farmer and fishermen working in factories that we would have labeled "sweatshops" not so long ago (for a minimum wage hopefully) and former (US) textile workers not working at all.
  23. Stupid Comment[ Go to top ]

    I find it quite strange to assume that these people would otherwise not have "any work". I don't happen to live in the US, but afaik, the policy of walmart to import textiles from China has ultimately pushed all its competitors to do exactly the same. So we now have former farmer and fishermen working in factories that we would have labeled "sweatshops" not so long ago (for a minimum wage hopefully) and former (US) textile workers not working at all

    The point is that nobody forces them to work in the factory. If a factory opens up paying 25% more than your current salary and you are a happy fisherman, you might decide to stay a fisherman. However, if you are an unhappy fisherman you might decide to become an unhappy factory worker but have a larger salary. Nobody forces them to work at the factory. Now if the government forced people to work at the factories that would be different.

    Furthermore, try not to complain about both sides at the same time.
     1. It's not fair, people overseas do not make enough and need better jobs.
     2. It's not fair, american's don't have jobs because all the good jobs are going overseas.

    Which is it?
  24. Even more stupid....[ Go to top ]

    The point is that nobody forces them to work in the factory. If a factory opens up paying 25% more than your current salary and you are a happy fisherman, you might decide to stay a fisherman.

    I tried to stay away of this discussion, but I can't. Your comments are just way too much out of reality - and just as arrogant and worng. You talk about choice just like if the farmar/fisherman couls judt take a plane back home if the job isn't good.

    Please don't talk about what you do not understand ou know NOTHING about. The real story is always like:

    1. The sad farmer/fisherman is called to work in a far, far away factory (countries like China and Brazil are just as huge as US, but transportation is sometimes very difficult and expensive). He is convinced by promises of bigger salaries, money, better life etc.;

    2. When he gets to the factory/plant/mine/whatever, reality is very different. But he IS FORCED to work to pay the trip to the job - and to be able to pay the trip back (wich will always cost A LOT more than he can pay);

    3. He is FORCED to buy food/medicine/etc. from his employer - such jobs are in far places where there are no stores that do not belong to the employer;

    4. He is trapped forever - he actually gets no real payment at all, and his debts never stop to grow. This is, indeed, modern slavery.

    I really cannot believe that in these days there are still people that do not know that things happen like that. You don't need to go to China or Brazil - this kind of slave work happens even in NY basements (sweat shops).

    So, again, DO NOT express yourself about this subject before reading a little (or just looking around).
  25. good breakdown...[ Go to top ]

    Wow, how fucked up do you have to be to bring up the Holocaust in a comparison?

    well, that's exactly what the Godwin law predicts:

    "As an online discussion grows longer, the probability of a comparison involving Nazis or Hitler approaches one"
  26. Agree to disagree[ Go to top ]

    I have a simple question though. How do we make our web-applications fast, secure, responsive like Walmart.com with Java/J2EE. I know it can't just be a simple, "buy a faster machine or increase the size of the cluster".

    Even the serverside.com, which I assume is a 100% pure Java system seems slow and unresponsive(Can't get to the techtalks again).

    Has walmart or anybody created webapps that can give you the speed of a php/python/LAMP style application under serious load or what does walmart.com do that others don't.

    It seems like Jikes was used heavily and I noticed the blurb about the 100 node cluster.
  27. Agree to disagree[ Go to top ]

    I have a simple question though. How do we make our web-applications fast, secure, responsive like Walmart.com with Java/J2EE. I know it can't just be a simple, "buy a faster machine or increase the size of the cluster".Even the serverside.com, which I assume is a 100% pure Java system seems slow and unresponsive(Can't get to the techtalks again). Has walmart or anybody created webapps that can give you the speed of a php/python/LAMP style application under serious load or what does walmart.com do that others don't.It seems like Jikes was used heavily and I noticed the blurb about the 100 node cluster.

    I know first hand several sites using Tomcat handling 10million+ page views a day, so just because you don't know, doesn't mean it doesn't exist. PHP can scale well, since Yahoo uses it, but saying pure java systems don't scale is just totally wrong. Does PHP provide distributed transaction capabilities? Does PHP handle SOAP webservices + distributed transactions?

    let's focus on using the appropriate tool for the job and not make generalizations that have been consistently proven wrong. there are many ways to scale an application, but the key is knowing which techniques to use.

    peter
  28. Wow, just asking a question[ Go to top ]

    I don't know how you came up with so many assumptions based on my comments, amazing really. My question was purely used to gather information. I know java webapps can get there, I am just wondering the hows?

    "I know first hand several sites using Tomcat handling 10million+ page views a day, so just because you don't know, doesn't mean it doesn't exist."

    This is great, but what are the tradeoffs of using one JVM over the other and can Resin generate similar performance?

    "PHP can scale well, since Yahoo uses it, but saying pure java systems don't scale is just totally wrong."

    Where did I say that? Generally, most Java consultants will try to solve speed issues, by buying faster hardware, sometimes this approach doesnt work. Are there other solutions?

    "Does PHP provide distributed transaction capabilities? Does PHP handle SOAP webservices + distributed transactions?"

    SOAP libraries, is that really an argument? Yahoo uses a REST architecture, which probably will gain momentum over SOAP.

    "let's focus on using the appropriate tool for the job and not make generalizations that have been consistently proven wrong. there are many ways to scale an application, but the key is knowing which techniques to use."

    That is the point of asking questions, as I have done.
  29. Sorry about that[ Go to top ]

    I don't know how you came up with so many assumptions based on my comments, amazing really. My question was purely used to gather information. I know java webapps can get there, I am just wondering the hows?"I know first hand several sites using Tomcat handling 10million+ page views a day, so just because you don't know, doesn't mean it doesn't exist."This is great, but what are the tradeoffs of using one JVM over the other and can Resin generate similar performance?"PHP can scale well, since Yahoo uses it, but saying pure java systems don't scale is just totally wrong." Where did I say that? Generally, most Java consultants will try to solve speed issues, by buying faster hardware, sometimes this approach doesnt work. Are there other solutions?"Does PHP provide distributed transaction capabilities? Does PHP handle SOAP webservices + distributed transactions?"SOAP libraries, is that really an argument? Yahoo uses a REST architecture, which probably will gain momentum over SOAP."let's focus on using the appropriate tool for the job and not make generalizations that have been consistently proven wrong. there are many ways to scale an application, but the key is knowing which techniques to use."That is the point of asking questions, as I have done.

    my apologies for jumping on that comment, but it was phrased in a negative tone. I know Yahoo does use SOAP, along with a ton of other stuff. The number of technologies Yahoo uses varies quite a bit, so it's not just REST.

    What makes a website scale is the individuals building and maintaining the system, not the technology. A better question to ask might be, "what types of problems are unique to large scale/high volume websites, which are different from small websites?" A follow up question to that might be, "what techniques are used to solve that class of problems?"

    Once you understand the nature of the problem, the original questions might appear irrelevant. Although there are difference between Sun, IBM and JRockit, what matters with high volume is understand the nature of the loads and then running several hundred benchmarks to measure the optimal settings. I've done that several times in the past and there is no magic bullet. The first thing is you have to understand what the traffic pattern is in terms peak/average concurrent requests and the type of requests. Once you know those details, it's just a matter of building good stress tests and repeating the tests until the system meets the requirements.

    peter
  30. PHP vs Java... oh no not again[ Go to top ]

    <blockquoteHas walmart or anybody created webapps that can give you the speed of a php/python/LAMP style application under serious load or what does walmart.com do that others don't.
    ...
    PHP can scale well, since Yahoo uses it, but saying pure java systems don't scale is just totally wrong.
    How can people compare these things at all?
    With PHP-like solutions, everything is always stored in the database. Which is indeed good, because it's simple.
    But
    1. database will become a bottleneck sooner or later
    2. there's no way to do smart caching in the middle tier; nor one could have any other 'application-wide' resources, pools or something alike.

    With Java, most "speed" of the application comes from the fact that wisely applied caching at various levels (provided that memory prices are now hilariously low) can improve application response time by an order of magnitude. When everything is served directly from memory, at Linux 2.6 with Java 1.5 I've seen latencies such as 1-2 ms per quite complex pages.

    This relieves database from unnecessary hits and allows further improvements, such as speading this cache across the cluster.
    Of course, such solutions are far more complicated (with all these "stale data" issues) than PHP, but they are also far more efficient in achieving higher performance on the same hardware, and also more scalable in the long run than PHP ones.
  31. Some PHP applications cache data in file system, probably it is not as fast as local memory, but it is possible to tune OS level cache for files. google must help to find cache implementations for PHP too. I am not PHP fan, but I think it is better to advocate JAVA without discrediting PHP.
  32. PHP vs Java... oh no not again[ Go to top ]

    Some PHP applications cache data in file system, probably it is not as fast as local memory, but it is possible to tune OS level cache for files. google must help to find cache implementations for PHP too. I am not PHP fan, but I think it is better to advocate JAVA without discrediting PHP.

    It is only stating fact. Caching in file system is going to be hugely slower than direct database access. Cacheing has always been an issue with PHP. It is possible to cache data in session scope, but that is obviously not scalable. PHP application-level cacheing is possible, but has to involve a lot of messy shared memory handling.
  33. PHP vs Java... oh no not again[ Go to top ]

    http://www.sitellite.org/docs/Cache/Cache.html
    There is wrapper for BDB, do you use better cache with JAVA than BDB with PHP ?
  34. PHP vs Java... oh no not again[ Go to top ]

    http://www.sitellite.org/docs/Cache/Cache.htmlThere is wrapper for BDB, do you use better cache with JAVA than BDB with PHP ?

    Yes, of course. With Java you can very easily store data in application scope, and this will work no matter what database you work with. No need for external code or add-on tools.
  35. my post...[ Go to top ]

    ... was just as off-topic Bob, but at least your post was hilarious ! Good find ;)
  36. Boxing, etc.[ Go to top ]

    He, he, he... I see you found the boxing commercial.

    I actually do Thai boxing; the commercial was all about having fun, getting some cash and all the Chinese food I could eat during a tough 9-hour shoot.

    About Thai boxing: http://eugeneciurana.com/musings/muay_thai_cert.html

    Have an awesome week,

    Eugene
  37. etc, etc[ Go to top ]

    Alain
    =====
    Thanks for the Article :)

    Mike
    =====
    Right, they aren't "forced" to work there with a gun to their head per se, but rather due to lack of choice. Viva la WTO.
    Capitalism isn't all bad, but rampant capitalism is.
    It is quite easy to say "ooooh shut up" when it isn't your immediate family involved, and you don't see the atrocities in first person. Ignorance is bliss.
    Some of us who accept to acknowledge "humanity" will continue to boycott these corporations.
    Oh yeah, and I won't even get started on Microsoft ;)

    Eugene
    =====
    Muay Thai is awesome. Got to see some intense demonstrations while in Phuket (i myself practice gung-fu).
    Also, excellent article. It's good to see a thought-out approach, and I'll be sending this thread to my colleagues here at work :)
  38. Just curious. In my experience, management would view this upgrade as time wasted and too risky. Instead they’d say we should focus on developing their new features/requirements and would laugh at "programmer demand to use the new language features" as a reason for an upgrade.

    "This would result in short-term impact to the production site but non-maintainable or brittle code in the long term."
    Management would push for the quickest route so we could develop their new features/requirements.

    So how did you convince management to go along with this? Did you sneak this in with another upgrade?
  39. Greetings,

    The appropriate folks carried out an analysis of the performance benefits in Java 5 vs. Java 1.4.x and prototyped that. The numbers were extrapolated and the plan that I described in the article was implemented.

    Management was receptive to the idea; it's all about common sense. If you do something risky, have a backup plan. Ours, as the article states, was to roll-back to 1.4.x if things broke in production. They didn't, we're on Java 5, life's good. The programmers are using the new Java 5 features nowadays.

    Cheers,

    E
  40. Greetings,The appropriate folks carried out an analysis of the performance benefits in Java 5 vs. Java 1.4.x and prototyped that. The numbers were extrapolated and the plan that I described in the article was implemented.

    ***shameless sales plug mode on***

    So you upgraded to Java 5 mainly to get the performance boost you find using the new version of Hotspot? Why not switch to JRockit 5 instead, which performs even better ;-)

    Cheers!
  41. How did you convince management?[ Go to top ]

    Fill us in....what are these atrocities that you have seen first hand?
  42. Just curious. In my experience, management would view this upgrade as time wasted and too risky. Instead they’d say we should focus on developing their new features/requirements and would laugh at "programmer demand to use the new language features" as a reason for an upgrade.

    "This would result in short-term impact to the production site but non-maintainable or brittle code in the long term."
    Management would push for the quickest route so we could develop their new features/requirements.

    So how did you convince management to go along with this? Did you sneak this in with another upgrade?
    Coming from the opposite angle, what you would need to tell management is how nasty things get when you get too far behind and that the longer you wait the harder it is to upgrade.

    I'm in a situation where there was no time for upgrades that didn't improve business functionality. Now we're using an dated tool kit (websphere and vaj 3.5) with very little documentation and no support. In order to keep up with newer technologies that aren't compatible with the environment a lot of costly work has be done to reimplement things like SOAP messaging and ejb.

    Using 3rd party libraries/tools (often open source) is essential and not keeping at least near the most recent makes this very difficult.
  43. Just curious. In my experience, management would view this upgrade as time wasted and too risky. Instead they’d say we should focus on developing their new features/requirements and would laugh at "programmer demand to use the new language features" as a reason for an upgrade."This would result in short-term impact to the production site but non-maintainable or brittle code in the long term."Management would push for the quickest route so we could develop their new features/requirements.So how did you convince management to go along with this? Did you sneak this in with another upgrade?

    Really? That's sad. We dont have problems in my company doing overhauls of code to improve development. We simply state, "Listen, xx amount of hours now is much less than the amount of time to maintain this code in the future. It will speed up our development and provide us flexability we havent had before." That seems to work rather well. Its kind of my speciality actually. Take existing code and refactoring it to provide architectural, performance and feature improvements. Just implementing the features could be faster, but implementing features on top of that might not. Refactoring is good.

    For people who aren't so lucky convincing their managers to switch to java 5, I would just point them to one of Sun's press releases. Shouldn't that be all it takes? :)
  44. this place's focus is new feature development & quick time to market. it's common sense to us engineers the importance of maintainability but management takes the opposite view in every shop I have worked. if there is a quick & dirty option & a maintainable option that takes a larger initial time investment, quick & dirty always wins with them.

    Refactoring doesn't exist to those folks. if it is working, they want it left alone & new requirements worked on. you can try to add refactoring to time estimates but it will get cut out when they squeeze your timeline.

    it is sad that more places aren't like walmart with a longer term vision.

    how large was the team that worked on this upgrade?
  45. How did you convince management?[ Go to top ]

    I'm sure it depends on the breadth of many factors, primarily the size of the organization in a whole, what it see's as it's futures in the online market... etc.

    The performance, scalable architecture and all the other benefits that can be gained when keeping up-to-date with the latest of the software/hardware trends could then make perfect sense.

    I'm definitely with TriggTrigg, in that most of the companies I've worked for in the past simply want things "done", and enhancements added... not giving a care in the world for the future.
    Like talking to a kid in a candy store about the benefits of vegetables ;)
  46. token comments[ Go to top ]

    TriggTrigg... good question. Being able to validate a migration of this magnitude is hard work, and is "definitely" not going to be put on any PRD's by most management.
    Assuming the Walmart engineering and technical management team is as large as one would assume, doing analysis on performance gains seems to be the road to take.

    Eugene, if you could make available, what white papers did your team use to gather data for performance gains... or did you do your own in-house benchmarks ?

    As for the off-topic topic... being one with friends and family in a "third world" countries and seeing what our overseas market is doing to communities... I find it necessary to throw in my share of knowledge.

    For example, if you're family worked at a Coca Cola bottling plant in Columbia, and you saw a thread about one of your favorite programming languages, with Coca Cola mentioned... you'd probably throw in your 2 cents too.
    However, you probably don't have family or friends who work there... you're probably not aware of the situation... so, you probably wouldn't post anything out of the ordinary... and, end up questioning someone who did post some facts about Coca Cola that were obviously off-topic, but relevant to life regardless.
  47. token comments[ Go to top ]

    Eugene, if you could make available, what white papers did your team use to gather data for performance gains... or did you do your own in-house benchmarks ?

    +1
    Please! Could you show a summary of the numbers about performance?

    Thanks!
    Joserra
    Modelos de Negocio-sw. Libre
  48. I wish Walmart was on .NET[ Go to top ]

    Then I could hate them more ;)
  49. Contribute to Java community[ Go to top ]

    Its time for Walmart to contribute to Java and Open source community. At least in terms of documentation on the setup etc etc.
  50. Architecture docs?[ Go to top ]

    Can you put the architecture document online? If not can you discuss how Java 1.5 made the application faster?
  51. Architecture docs?[ Go to top ]

    Can you put the architecture document online? If not can you discuss how Java 1.5 made the application faster?

    Howdy,

    I can't go into details for various reasons. In general, we keep an eye on:

    * Thread issues in the JVM -- lots of profiling
    * Wise use of thread priorities
    * Use of every JVM switch that we can think of that will help us manage memory more efficient
    * Caching where possible (what's the slowest part of fetching data when you have millions of items in your catalogue?)

    Also, assume that a dedicated team of engineers' sole job is to ensure that things are working smoothly.

    If you have access to the Hotspot JVM vendor's engineers, use them. They'll teach you a lot about how to tune things. It helps if you buy a lot of servers from the same vendor.

    Here's something else that you may want to check out:
    http://blogs.sun.com/roller/resources/watt/jvm-options-list.html

    Cheers,

    E
  52. Architecture docs?[ Go to top ]

    Apart from various caching strategies, the biggest gains are to be had from garbage collection tuning. There are several thousand articles out there that speak to it. To gain visibility into the garbage collection characteristics of your app, you could enable several switches (verbose:gc, PrintGCTimeStamps, PrintGCDetails, PrintGCTenuringDistribution etc.) and jvmstat/visualgc (free tools) provide you with a visual view. One additional reason to consider a move to jdk1.5 is the convenience of having "always-on" VM instrumentation (and Jmx etc.) as part of the core library.
  53. Just curious, the article says that they are using Apache + Tomcat. But netcraft site is showing that they are using Microsoft-IIS/6.0.0 (on Solaris 8 ?):
    http://uptime.netcraft.com/up/graph?site=http://www.walmart.com&probe=1

    Could you explain this ? Or netcraft os not showing the correct info ?
  54. Just curious, the article says that they are using Apache + Tomcat. But netcraft site is showing that they are using Microsoft-IIS/6.0.0 (on Solaris 8 ?): http://www.walmart.com&;probe=1">http://uptime.netcraft.com/up/graph?site=http://www.walmart.com&;probe=1</a>

    Could you explain this ? Or netcraft os not showing the correct info ?

    It's not unusual at all. Consider:

    1) Usually in a large web environment, the TCP connections are actually being handled by a HA load balancer pair

    2) The load balancers are typically sitting immediately in front or in back of a firewall, or sometimes even in between.

    3) Some sites load balance (spray) across a large number of web servers; IIS is a reasonable choice here because it has good SSL support (walmart.com is an ecommerce site).

    4) Most sites like this use different pieces of software to implement different parts of the site. For example, it wouldn't be surprising to see a portal product being used to handle parts of the site. We've seen Vignette used a lot for things like this, even if the site isn't a traditional "portal". Microsoft also has a portal product.

    5) If that doesn't make the netcraft stats wacky enough, J2EE apps can implement a simple filter that strips (or replaces) the server info out of the HTTP responses, and firewalls can be set up to do the same. I've seen the code that one of our customers uses to do this (to confuse hackers).

    Peace,

    Cameron Purdy
    Tangosol Coherence: Clustered Shared Memory for Java
  55. These guys worked really hard (especially Subir) to make this a very smooth upgrade. It takes lots of hard work and dedication to upgrade the web platform for the world's largest retailer.
    I know this because I was managing the team at the time. Thanks, Eugene, for this great article.

    Cheers,
    Armond
  56. Eugene, if you could make available, what white papers did your team use to gather data for performance gains... or did you do your own in-house benchmarks ?
    +1
    What version of Tomcat are you using ?
    +1


    Eugene, could you enlighten us ?
  57. Eugene, if you could make available, what white papers did your team use to gather data for performance gains... or did you do your own in-house benchmarks ?
    +1
    What version of Tomcat are you using ?
    +1Eugene, could you enlighten us ?

    We did our own in-house benchmarks. As you can imagine for a site this big, there are a lot of monitoring/profiling tools in place, both for testing/staging and production. The guys working on this spent a lot of time analyzing these monitoring tools.

    We also built some tools based on JMX technology. We started with the JVM Vendor's stuff then customized it to monitor what we actually work on.

    About Tomcat: no, I can't talk about specific versions/releases; those are the house rules for the article's author.

    A couple of our platform guys are following this thread and I'll defer to them to answer this if they think it's appropriate. I can tell you that Apache and Tomcat are recent stable releases.

    Cheers,

    E
  58. Awesome. Thanks Eugene, and definitely a BIG congrats out to you and your team !
  59. any more details on backend?[ Go to top ]

    I'm curious how much of it was done in the database via stored procedures and how many servers it uses to be "massively parallel"?
  60. any more details on backend?[ Go to top ]

    I'm curious how much of it was done in the database via stored procedures and how many servers it uses to be "massively parallel"?

    I can't discuss our actual database server configuration, vendor, etc. Not how many processors per server, or how many servers either. Sorry, those were the rules that my director gave me so I could publish the article, i.e. no proprietary or vendor information. Think of that database set up as one of the biggest ones commercially available in the world. There are very few that big. And the "massively parallel" part is not an exaggeration. That ought to narrow your Google search a bit more.

    Now, as far as stored procedures are concerned, you're right: there's quite a bit of reliance on stored procedures. We use a programming language / SQL technology from a commercial vendor; a lot of the business logic is implemented that way. There are no ad hoc queries or dynamic SQL involved.

    I hope this helps, and thanks for reading the article,

    E
  61. any more details on backend?[ Go to top ]

    Walmart is a huge IBM mainframe shop along with Teradata.
  62. What exactly does java do in this case(-version:release). Is it only a flag for java to use 1.4 instead of 1.5 or it uses 1.5 vm but 1.4 classes and jars.