Discussions

News: Problems on TSS

  1. Problems on TSS (77 messages)

    Many of our users have informed us that TSS is "having problems." We're aware of them and what causes them - and most importantly, we're working on fixing them.

    For one thing, some queries are very expensive. Because of the way the data is pulled from the datastore, the database server is being loaded down, which causes problems in the application itself. This has performance implications, obviously, and causes application errors as well.

    Another problem, one slightly more insidious, is that contributed posts are being lost. Most users will see this as their comments not being posted to the site. The cause for this may be related to the database problem mentioned earlier, but I don't think it is.

    These are the two most severe problems TSS has right now. Both are related to data, not UI (although we're aware of some UI issues too), both have been prioritized highly, and we're on track for fixing both very soon - and there's an effort to upgrade TSS off of its current codebase in a few months, which should bring the entire application up to date.

    In the meantime, we're dedicated to bringing you the quality content that TSS has always tried to offer, and we hope you'll be patient while we iron out the issues. In fact, I'd love to hear what features you'd like TSS to have, in the future. What's your vision?

    If you'd like to email us to make sure your thoughts on this are aired, email me at jottinger at techtarget dot com and I'll do my best.

    Threaded Messages (77)

  2. Problems on TSS[ Go to top ]

    Another problem, one slightly more insidious, is that contributed posts are being lost. Most users will see this as their comments not being posted to the site.

    I haven't pin-pointed the exact criteria for failure but for me, it has something to do with apostrophes. When by post doesn't show up, I go back and get rid of apostrophes and try again. Eventually it posts.

    It seems to be something like an even number of apostrophes is OK but odd numbers are not. This would suggest an issue with encoding text for DB inserts.
  3. Problems on TSS[ Go to top ]

    Two 'apostrophes' attempt failed three times. Removing one.
  4. Problems on TSS[ Go to top ]

    Two `apostrophes` attempt failed three times. Removing one.
  5. Problems on TSS[ Go to top ]

    James, sadly, I have to disagree with you. I post on the site all the time (obviously!) and I can get it to break on a regular basis regardless of the apostrophes or lack of them. I know what and where the central problems are; apostrophes are entirely incidental.

    The fact that you've been able to post to collect this data is, sadly, purely anecdotal and incidental. We're on the problems. As soon as we have them fixed, you'll notice - if only for the whoop of triumph from the various locations of TSS employees. :)

    Folks, we appreciate bug reports - really and truly. However, be very careful trying to do armchair analysis. We really do know where the problems lie, and we're working on them. (In fact, we have an internal postmortem on why the problems exist in the first place.) It's easy to see anecdotal events and say "It's the plus sign!" or "It's the apostrophes!"

    It's nothing quite so simple - and, conversely (and perversely), it is fairly isolated. It's just that fixing the bug involves some interesting work.

    That said, again - any and all comments are welcomed.
  6. Problems on TSS try 8[ Go to top ]

    Here is my 8th attempt. I don't care how many times I have to submit this I will get one through!!

    I love ranting. Maybe the anti-Rant filter is reading my mind and blocking me.

    I will meditate before using TTS, to try to calm and clear my mind. Then and only then I might have a chance of making at past the anti-Rant filter.

    Even if I do make it past the mind reading anti-rant filter I will probably get an RMI error, then I will super pissed, makeing me want to rant, and inturn I will be block.

    I'm screwed.....


    Also the system is more screwed up then you think. I post a message and it never shows up on the boards, but then I try to post it again, and it says I have already posted it. That means you go my post and know that I posted, but you can't display it.

    Also everytime I get a database error it is a kodo error. If I was Patrick I would be mad that you are giving his product a bad name.

    and don't even get me started on the RMI errors. Don't try to blame those on the database!
  7. Problems on TSS[ Go to top ]

    Hello Joe,

    Maybe you should consider open-sourcing the TSS code), so we can help you out ? Many Java developers (including myself) could also learn how to write such a very nice collaborative Java environment !

    Good luck in your Quest.

    Stephan
  8. OS-TSS[ Go to top ]

    Maybe you should consider open-sourcing the TSS code

    Now there's an interesting idea, you'd have to grit your teeth for the first few weeks in case people found nasty holes and backdoors though.

    -John-
  9. Problems on TSS try 8[ Go to top ]

    yeah ranting is great, we should find smarter rant ways.
  10. Problems on TSS try 8[ Go to top ]

    Also the system is more screwed up then you think. I post a message and it never shows up on the boards, but then I try to post it again, and it says I have already posted it. That means you go my post and know that I posted, but you can't display it.


    Not quite. Tapestry is "to blame" for this bug, which itself is a consequence of the datastore problem. Once you've submitted the post, Tapestry knows you've done it, and tries to prevent a "repost" caused by you just refreshing the original page, so it's saying - quite logically - that you've already posted this!

    That wouldn't matter if the data weren't getting lost and there wasn't a need to repost the same data.

    As far as database errors, well, like I've said, we're aware, and so is Patrick. :)
  11. Tokens suck[ Go to top ]

    Also the system is more screwed up then you think. I post a message and it never shows up on the boards, but then I try to post it again, and it says I have already posted it. That means you go my post and know that I posted, but you can't display it.
    Not quite. Tapestry is "to blame" for this bug, which itself is a consequence of the datastore problem. Once you've submitted the post, Tapestry knows you've done it, and tries to prevent a "repost" caused by you just refreshing the original page, so it's saying - quite logically - that you've already posted this! That wouldn't matter if the data weren't getting lost and there wasn't a need to repost the same data. As far as database errors, well, like I've said, we're aware, and so is Patrick. :)
    The TSS issues clearly show that tokens suck. Web-layer tokens know nothing about what happens with real data. On the other hand, it seems that TSS (or Tapestry itself) goes beyond regular tokens to track reposts. Looks like you run a digest on posted message to verify that the same exact content was not yet posted, because I cannot repost even if I selected the same topic again from "News" page, or even if I used a different browser.
  12. Tokens suck[ Go to top ]

    The TSS issues clearly show that tokens suck. Web-layer tokens know nothing about what happens with real data. On the other hand, it seems that TSS (or Tapestry itself) goes beyond regular tokens to track reposts. Looks like you run a digest on posted message to verify that the same exact content was not yet posted, because I cannot repost even if I selected the same topic again from "News" page, or even if I used a different browser.

    Well, I can accept your statement on that. I don't have a value-judgement on it myself; obviously, a perfectly-working system is better than an imperfect system.

    I can think of a few ways to validate double-posting; how would you suggest it be done?
  13. Tokens suck[ Go to top ]

    I can think of a few ways to validate double-posting; how would you suggest it be done?

    Check uniquity in database: store generated unique key in a hidden field and save it in database.

    Nebojsa
  14. Tokens suck[ Go to top ]

    Use a checksum stored in the database and stop using web-layer approaches to data(base) domain issues. Most of the framework stuff is a load of horsefeathers anyways. It's no irony that TSS is being discredited by the tapestries that it purports.
  15. Token implementation[ Go to top ]

    On my framework (http://twiff.sf.net/) I have a separate database for just the presentation tier which tracks double clicks. It seems to work when I try it.
  16. Token implementation (take 2)[ Go to top ]

    take 2 on the reply.
    On my framework (http://twiff.sf.net/) I have a separate database for just the presentation tier which tracks double clicks. It seems to work when I try it.


    Sounds like you could be subject to the same problem. If your presentation database stores that someone submitted a page, but that data was rolled back by your main database, then you will have the same problem TTS is having.

    Plus keeping track of presentation level tracking data in a database sounds like a bad idea.
  17. Token implementation[ Go to top ]

    Plus keeping track of presentation level tracking data in a database sounds like a bad idea.

    You might be right on that part, but I haven't found of any way of doing it except to store the tracking data in a session. Though it might be the normal place of putting things, I find that people sometimes forget to clear things from the session or rely on it too much that it may impact their application speed when they do clustering (in which case it will incur serialization costs as it will be stored in a database or some other shared memory pool.

    Mind you the implementation I wrote up in the sample uses an inmemory HSQLDB as well.

    (I must be one of the lucky few, I still haven't seen this error you guys have been talking about).
  18. Token implementation[ Go to top ]

    Mind you the implementation I wrote up in the sample uses an inmemory HSQLDB as well.

    Good point. Using an inmemory database in this way does make more sense.

    How do you handle the problem of a transaction failing in the main database. Do you have code that automatically corrects the tracking information in the presentation database? If so is this done manually or handled by your framework?
  19. Tokens suck[ Go to top ]

    private byte postAttempts=8;
    Use a checksum stored in the database and stop using web-layer approaches to data(base) domain issues. Most of the framework stuff is a load of horsefeathers anyways. It's no irony that TSS is being discredited by the tapestries that it purports.

    I don't quite understand the last sentence. The site is being discredited by the... tapestries (?) it "intends?"

    I'm assuming the "tapestry" is a reference to the UI, considering that the current codebase is in Tapestry.

    But I don't think the site is being discredited; it's simply got a few (major) bugs. I've tried to explain why those bugs exist, and discuss a little of what they are. I don't quite understand the "discredit" part. Can you explain further?
  20. Tokens suck[ Go to top ]

    In order to clarify this, please enroll yourself in English Comprehension and Critical Thinking Skills courses available at your local community college.
  21. Tokens suck[ Go to top ]

    how would you suggest it be done?

    Use some kind of DB cashing and check the post against the cache. I do this in TopLink all the time.
  22. Tokens suck[ Go to top ]

    I can think of a few ways to validate double-posting; how would you suggest it be done?

    FWIW the way I would do it is generate an id when you click "reply". Then you just make sure that gets used only once. I guess you have to store them in a sorted set of bounded size, to make sure you have got the memory under control.
  23. Tokens suck[ Go to top ]

    Just redirecting after post and disabling the submit button in javascript with a timeout to re-enable it in case of a network failure takes care of most double post scenarios without any server-side state handling.
  24. Tokens suck[ Go to top ]

    Joe Ottinger: I can think of a few ways to validate double-posting; how would you suggest it be done?
    Webapps are not monolythic desktop apps, they are client/server apps. Web UI is just a facility to present output data to a user and to accept input data, that is it. Unlike desktop application, which has only one UI, tighly coupled with model, webapp can have different clients and UIs, web UI is only one of many. Ideally, UI should not bother validating user input. Its purpose is to collect input data and feed it to the backend. It is the backend's task to verify the validity of data, to ensure the consistency of database, etc. Model bleeds into UI as a help for users to input data (reference tables, comboboxes) and to improve performance (caching, request short-cirquiting, etc). Ideally, this helper functionality should be transparent between dumb stateless UI and a backend model.

    Back to original question. I would say, that in world of clean separations of concerns a webapp should not check for double posting; back end should. I am not sure does TSS run a checksum on the posted comment or not, but this is a nice thing to have to identify duplicate. Seems that it does, since I cannot post the same content even if I start a new session in a different browser. Current problem on TSS is that information about already posted comment is pulled not from the database, but from a web-layer object, which is not informed about database failure.

    Instead of verifying the content one can verify the object ID (that is, database PK). In this case it is easier to create a new PK right before presenting an empty form to a user. Only PK is verified. If comment with the same PK is found, it is simply updated. This is a cheap solution to edit a comment if it was posted incorrectly or a poster wants to change a word or two.
    Geert Bevin: Just redirecting after post and disabling the submit button in javascript with a timeout to re-enable it in case of a network failure takes care of most double post scenarios without any server-side state handling.
    * If result page is different from submit form, then disabling submit button right after form is submitted is a cheap protection from resubmits, caused by trembling fingers.
    * Redirecting after post protects from implicit resubmits, but does not protect from explicit resubmits when a user navigates back to submit form. There are two choices for submit form: (1) either to cache it in the browser and show previous content with text that was just submitted, or (2) to make the form non-cachable and display "form has been submitted" message when a user navigates to the form. I prefer to use option (2) in my apps, but for a message board this is unacceptable especially if a user cannot change the posted message. Apparently TSS verifies the content which is posted, and if it detects that the same content is reposted, it rejects the posting. We see now that this does not work well when database does not report failure to the UI layer.
  25. Problems on TSS[ Go to top ]

    We really do know where the problems lie, and we're working on them. (In fact, we have an internal postmortem on why the problems exist in the first place.)

    I know that you don't want to tell everybody what the problem is, but maybe you should, since this web site is about server-side java. If the problem is one of design, then you should come out with it.

    BTW, I don't understand why you have scalability problems when your workload can scale horizontally over almost any number of servers using reliable multicast.

    Good luck.
  26. Problems on TSS[ Go to top ]

    private int m_cAttempts = 0x0D;
    BTW, I don't understand why you have scalability problems when your workload can scale horizontally over almost any number of servers using reliable multicast.

    When they were having database problems previously (e.g. April 12, according to my IM logs) they dropped down to a single app server instance, and limited its connection pool size to 10. Even at that level, the database processes were all hung up and timing out doing selects.

    Unfortunately, while it is tunable to a large extent, Postgres is not a self-optimizing database by any stretch of the imagination. To successfully use it with any amount of load, every query needs to be tested individually for an optimized plan (using "EXPLAIN"), and tiny variations in the queries can make gigantic differences in the plan, particularly in the use of indexes.

    I'm a big fan of Charles Miller, an Atlassian dude with a wicked sense of humor and an incredible ability with the English language (a language which I have been endeavoring to learn by reading his blog). Most of what I know about tuning Postgres come from him and others (like Floyd, Dion, Patrick and others who have lost lots of hair using Postgres). I found these blog entries from Charles useful:

    http://fishbowl.pastiche.org/2004/06/17/i_love_postgres
    http://fishbowl.pastiche.org/2004/10/15/no_swap_for_you

    From what I've seen, a very simple example of how to bring Postgres to its knees is to find two queries that do table scans that could easily be handled by an index; as Charles pointed out, this is way too easy to accomplish. Then join those two queries using something as simple as "where PK=FK" and the result will be a query that never completes (assuming that there is any significant amount of data in the database).

    If you have a Java app issuing that query, it will tie up a JDBC connection indefinitely. If you have a spider hitting a sequence of pages in parallel, each containing a link that causes that particular query to execute, then you have an instant site melt-down. Someone like William Louth (with his JDBInsight product) could probably identify the biggest SQL problems inside an hour.

    When this type of problem occurs, it will manifest itself in a number of ways, such as:

    - no connection available in the pool (they're all running queries that don't finish within the timeout period of the app server)
    - RMI error (if the query is being run down below an RMI call that has a timeout)
    - slow site (even with light load, the database is constantly working extremely hard to do nothing of value)

    At any rate, I'd like nothing better than to see them put in an architecture that is designed for horizontal scale. Right now, though, they've got to figure out what's killing the database.

    Peace,

    Cameron Purdy
    Tangosol Coherence: Clustered Transactional Caching
  27. Problems on TSS[ Go to top ]

    private long failedAttempts = 3;
    From what I've seen, a very simple example of how to bring Postgres to its knees is to find two queries that do table scans that could easily be handled by an index; as Charles pointed out, this is way too easy to accomplish. Then join those two queries using something as simple as "where PK=FK" and the result will be a query that never completes (assuming that there is any significant amount of data in the database).

    My favorite PosgtreSQL example:

    select * from foo as f
    where foo.n < 100

    is equivalent to

    select * from foo as f, foo
    where foo.n < 100

    If table foo has many rows it will take forever.

    Nebojsa
  28. Problems on TSS[ Go to top ]

    This is a typical case where the price for a DB like DB/2, Oracle or SQL Server would be worth.
    Postgresql was free to begin, but now it will be a more costly solution… Machines and Software are cheap…. Development and consultants not!
    With SQLServer I would just use the Profiler to analyze the workload and the correct indexes would be suggested and created!!

    Thats something that I've been learning with time... I'm tired of cheap solutions that turn into nightmares.

    Pedro Costa
  29. Problems on TSS[ Go to top ]

    protected BigInteger postAttempts=new BigInteger("15");

    Well, as I've stated elsewhere, PostgreSQL isn't the problem here. DB/2, Oracle, SQL Server... all would have the same problems, and for exactly the same reasons. The indexes are correct, from what I'm told (the developers DO know how "explain" works...) so index optimization would still have problems.

    Again: it's not the DB that's failing here. If it was, we'd have switched off a long time ago.
  30. A couple of humble suggestions[ Go to top ]

    Well, as I've stated elsewhere, PostgreSQL isn't the problem here. DB/2, Oracle, SQL Server... all would have the same problems, and for exactly the same reasons. The indexes are correct, from what I'm told (the developers DO know how "explain" works...) so index optimization would still have problems.Again: it's not the DB that's failing here. If it was, we'd have switched off a long time ago.
    Hmmm... So, the PostgreSQL DB has no issues, the application is fine, JBoss / Tapestry / Kodo are beyond reproach - but somehow, it just doesn't work. Suggestions:
    (1) Use JXInsight to pinpoint the problem; it's a wonderful tool that will show you most probably within minutes where all those queries are coming from
    (2) If (1) does not suffice, doesn't Kodo support Cameron's hashmap ? Can't you avoid a large part of the database hits just through some cache configuration work ? (see http://docs.solarmetric.com/manual.html#ref_guide_datacacheintegrations)

    PS: got this on my first attempt: It says it's all Bill's fault :-)

    Error posting message: org.jboss.tm.JBossTransactionRolledbackException: null; nested exception is: org.jboss.tm.JBossRollbackException: Unable to commit, tx=TransactionImpl:XidImpl[FormatId=257, GlobalId=sanchez.techtarget.com/101, BranchQual=, localId=101] status=STATUS_NO_TRANSACTION; - nested throwable: (kodo.util.FatalDataStoreException: java.util.NoSuchElementException: Timeout waiting for idle object); - nested throwable: (org.jboss.tm.JBossRollbackException: Unable to commit, tx=TransactionImpl:XidImpl[FormatId=257, GlobalId=sanchez.techtarget.com/101, BranchQual=, localId=101] status=STATUS_NO_TRANSACTION; - nested throwable: (kodo.util.FatalDataStoreException: java.util.NoSuchElementException: Timeout waiting for idle object))

    Second attempt: no, it's Patrick's fault!

    java.rmi.ServerException: RuntimeException; nested exception is: kodo.util.DataStoreException: java.util.NoSuchElementException: Timeout waiting for idle object

    Third and following attempts: It's actually Joseph "you don't know me"'s fault:

    This message has already been posted once. This error results if you click on submit twice, or if you use the back button and submit the message again.

    Sixth attempt: I hate mankind.
  31. A couple of humble suggestions[ Go to top ]

    private int postAttempts=2;
    Hmmm... So, the PostgreSQL DB has no issues, the application is fine, JBoss / Tapestry / Kodo are beyond reproach - but somehow, it just doesn't work. Suggestions:
    (1) Use JXInsight to pinpoint the problem; it's a wonderful tool that will show you most probably within minutes where all those queries are coming from
    (2) If (1) does not suffice, doesn't Kodo support Cameron's hashmap ? Can't you avoid a large part of the database hits just through some cache configuration work ? (see http://docs.solarmetric.com/manual.html#ref_guide_datacacheintegrations)

    Nobody said the DB had "no issues," nobody said the libraries were "beyond reproach." We know there are errors. We know where the errors are. We're also not pointing fingers, as it's not really worthwhile. I've also explained why.

    We could use JXInsight, I suppose; but at this point, like I said, we know where the queries are coming from. Without a doubt.

    As far as caching, there are requirements that void caching, so Coherence is unable to help. That's being addressed, too.

    It's not worth laying blame on anyone, nor is it worthwhile to say "It's JBOSS!" or "It's POSTGRESQL!" or "It's KODO!" or "It's TAPESTRY!" or "It's COHERENCE!" or "It's CHRIS!" or "It's JOHN!" Saying anything like that doesn't help the problem; it only attempts to pinpoint the problem in the past, and does nothing to address fixing the problem for the future.

    However, like I've said, we're addressing it. It will be fixed before the rearchitecture. In fact, it'll be fixed soon.
  32. Ahem[ Go to top ]

    Nobody said the DB had "no issues,"
    Except a certain J.O. in the post I was responding to:
    Well, as I've stated elsewhere, PostgreSQL isn't the problem here. DB/2, Oracle, SQL Server... all would have the same problems, and for exactly the same reasons. <snip>
    Again: it's not the DB that's failing here. If it was, we'd have switched off a long time ago.
  33. Ahem[ Go to top ]

    Nobody said the DB had "no issues,"
    Except a certain J.O. in the post I was responding to:
    Well, as I've stated elsewhere, PostgreSQL isn't the problem here. DB/2, Oracle, SQL Server... all would have the same problems, and for exactly the same reasons. <snip>Again: it's not the DB that's failing here. If it was, we'd have switched off a long time ago.

    Well, Alain, "not the problem here" doesn't mean the same thing as "has no issues."

    Alain... is there a personal problem here? If I've offended you, perhaps you and I should take it up in a different forum. This isn't the place for that rubbish. I don't know what your tone means in this context; it's coming across as very hostile.
  34. Logic[ Go to top ]

    Well, Alain, "not the problem here" doesn't mean the same thing as "has no issues."
    I'm just having a hard time following your logic. You said very clearly that PostgreSQL had nothing to do with the problems and that another RDBMS would behave just the same because it was being hammered badly by the application. The distinction you are trying to make between "having issues" and "being the problem" reminds me of Jesuit casuistic :-) You said the problems originate entirely from the way the application is written, therefore it's correct to say that PostgreSQL "isn't the problem". It may no be perfect as Cameron pointed out, but that has nothing to do with the core issue under discussion.
    Alain... is there a personal problem here? If I've offended you, perhaps you and I should take it up in a different forum. This isn't the place for that rubbish. I don't know what your tone means in this context; it's coming across as very hostile.
    There is nothing personal of course, but if you lecture me about my logic, you should expect I might disagree and point out where the logicals flaws are.
  35. Logic[ Go to top ]

    private long postAttempts=1;
    Well, Alain, "not the problem here" doesn't mean the same thing as "has no issues."
    I'm just having a hard time following your logic. You said very clearly that PostgreSQL had nothing to do with the problems and that another RDBMS would behave just the same because it was being hammered badly by the application. The distinction you are trying to make between "having issues" and "being the problem" reminds me of Jesuit casuistic :-) You said the problems originate entirely from the way the application is written, therefore it's correct to say that PostgreSQL "isn't the problem". It may no be perfect as Cameron pointed out, but that has nothing to do with the core issue under discussion.

    Casuistry is fun. But it's also distracting and silly. I wasn't using it, incidentally. I'm trying to pay attention to the major issues on TSS; I can't speak to the specific schema design, except to say that with the major issues at hand, the specific DB isn't the problem. (To wit: I do know of part of the schema design that's done rather poorly. However, with the problems with priority, such problems are not the cause for TSS' issues.)

    Do you see what I mean? I didn't say it had "no problems," but any problems it does have aren't directly related to the problems I mentioned in the thread.

    In any case, this is not constructive. I'm not interested in pointing fingers and saying "It's this!" or "It's that!"
    Alain... is there a personal problem here? If I've offended you, perhaps you and I should take it up in a different forum. This isn't the place for that rubbish. I don't know what your tone means in this context; it's coming across as very hostile.
    There is nothing personal of course, but if you lecture me about my logic, you should expect I might disagree and point out where the logicals flaws are.

    Of course there's nothing personal. That's why you quoted my home page, which has nothing to do with this thread or even this site, and that's why you're trying to expose logical "holes" that weren't "holes" in the first place?

    I'm afraid I just don't understand your point.
  36. Logic[ Go to top ]

    Attempt #10
    Of course there's nothing personal. That's why you quoted my home page, which has nothing to do with this thread or even this site,
    Huh ? I have no idea what you are even talking about here. Aren't you getting a little paranoid ?
    and that's why you're trying to expose logical "holes" that weren't "holes" in the first place?I'm afraid I just don't understand your point.
    They were and still are logical holes to me and I would have pointed them out just the same if they originated from somebody else. It's up to you to believe there must be some conspiracy to destroy your credibility.
  37. A couple of humble suggestions[ Go to top ]

    Joseph: "We could use JXInsight, I suppose; but at this point, like I said, we know where the queries are coming from. Without a doubt."

    Hi Joseph,

    I am sure you know where the queries are coming from after having sometime to investigate but I wanted to maybe add some insight to this thread that could be applicable to others especially as I see it a common thread with many production applications that are under performing.

    With many of our customers we have seen that before they started using JXInsight the testing focus was simply execute a single use case (under load) and capture the response times maybe logging some queries to a file to understand what are the poor performing SQL statements. Then whenever a change is made in the application all the test cases are executing individually and the times compared. If there is no large deviation then deploy into production.

    Unfortunately this simplistic approach has lead to many production systems grinding to a snails pace with operations staff forced into rolling back changes. When this is fired back at QA testing reports are produced showing that in the lab it performed acceptable. What went wrong?

    The biggest mistake made is not testing and analysing the behavior of transactions and SQL statements executed concurrently. Even when such testing is performed you must be careful not to focus solely on the slowest SQL statements because with concurrency the issue can be a specific concurrent transaction pattern or SQL statement that indirecly impacts the execution times of other SQL statements and transactions. Effort should be on understanding under what circumstances (concurrent transaction mixes) does the performance behavior of the system alter drastically from the norm. It is for this reason we provide two modes for component/database transaction analysis - profile and timeline.

    An article that explains this in a little more depth (but not as good as I would like) can be found on our website.
    http://www.jinspired.com/products/jdbinsight/concurrency.html


    Kind regards,


    William Louth
    JXInsight Product Architect
    CTO, JInspired

    "J*EE tuning, testing and tracing with JXInsight"
    http://www.jinspired.com


    PS: Alain and Cameron, thanks for the product recommendation it is appreciated.
  38. A couple of humble suggestions[ Go to top ]

    We know there are errors. We know where the errors are.

    Somehow this reminded me of
    There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don't know. But there are also unknown unknowns. There are things we don't know we don't know.

    ;-)
  39. Problems on TSS[ Go to top ]

    private BigInteger postAttempts=new BigInteger("8");

    Okay. PostgreSQL is getting HAMMERED, and while the queries might use some tuning, it's not the optimization of the queries that's the only problem: it's the fact that the queries are being made at all. It's something like querying for a count of records instead of storing that count in a tuple for quick retrieval - and even worse, instead of merely a count(*), it's actually retrieving and processing each record.

    Sounds like a trivial fix, doesn't it? But it's less trivial than it seems, for various reasons. I can't speak for the wisdom of the requirements - I don't know them all! - but really, if the fix were as trivial as people suggest it should be, it'd have been fixed by now.

    Again, it's not a matter of DB tuning (although it might be, I suppose) - it's a matter of what the database is being asked to do.
  40. Problems on TSS[ Go to top ]

    I think this whole TSS having issues is just cover for their new Anti-Rant Filter. Now *everyone* is having problems posting-- but really, there are no problems, TSS is working perfectly with the new Filter.

    The filter is so smart that it blocks the home page-- it knows you are there to rant, to it just intercepts you earlier.

    Jacob...
  41. Problems on TSS (anti-rant filter)[ Go to top ]

    Ah, if only it were true... an anti-rant filter would be a cool idea. Hmm, maybe we could train a bayesian filter...
  42. Maybe I'm wrong but...[ Go to top ]

    Problems of TSS
    1. Earlier mentioned data problems
    2. EJB
    3. Tapestry
    4. Poor forum design (for example viewing thread with hundeds of posts)
  43. Maybe I'm wrong but...[ Go to top ]

    Problems of TSS
    • Earlier mentioned data problems
    • EJB
    • Tapestry
    • Poor forum design (for example viewing thread with hundeds of posts)

    Wow. Um... I'm not sure where to go with this, so I'll dive right in.

    Data problems: these ARE the biggest issue with TSS. That's why I said they were.

    EJB: Hard for that to be the problem... TSS isn't based on an EJB architecture as far as I know. (I haven't examined the codebase - I'm a mere editor!) Having said that, maybe EJB is a core part of the architecture; if it is, the people who ARE looking at the problems in detail didn't mention EJB as being related to the two problems at all. Not once. They did, however, pinpoint the failures. When you have the smoking gun, chances are it's the murder weapon; you stop bothering to look for a knife.

    Tapestry isn't the problem either. Given the nature of the problem, Velocity, Freemarker, JSP, JSF -- all would have had the same problems. The UI mechanism may have issues, but in relation to the major errors, the UI is irrelevant.

    Forum design: well, we've been bouncing around lots of ideas for that. You say "there's a problem," suggesting viewing a thread with hundreds of posts. What's the exact nature of the problem? What would you do to improve it? What would you like to see?

    However, "forum design" isn't THE PROBLEM - and, in all honesty and humour, right now with TSS losing data, there won't be any new threads with hundreds of posts until we get the problem fixed. :)
  44. Problems on TSS[ Go to top ]

    some queries are very expensive. Because of the way the data is pulled from the datastore, the database server is being loaded down, which causes problems in the application itself. This has performance implications, obviously, and causes application errors as well.

    This is interesting news, considering that TSS is hardly an OLTP application (phone company billing system is an example of real OLTP). Nonetheless, the problems with TSS are reportedly related to database, not to web layer. Could this mean that Tapestry is so efficient indeed, that it managed to knock down the database instead of crashing from session overload? It would be really interesting if TSS had several frontends implemented with different frameworks. This would be a real-life test of "stateless" applications compared to "stateful" applications.

    But whatever the reason for lost postings is, the application should not report back to a user that a comment was posted successfully, if this is not true. Seems that TSS does not handle exceptions well enough.
  45. Lots of things in here!

    The problems are in data retrieval and database organization. It's not a matter of Tapestry being efficient (or not) - Tapestry is sort of the "cause" for the exceptions being propagated to the users, I guess, but that's not Tapestry's fault - the migration was never fully completed. The statefulness or not of the application isn't the factor here.

    It was suggested internally that we go through and fix things like the exception propagation, of course. But with a full rearchitecture on the horizon, some things are going to get put off. When the major problems get fixed, most exceptions will go away, which means whether they're reported to the user or not becomes irrelevant.
  46. But with a full rearchitecture on the horizon, some things are going to get put off. When the major problems get fixed, most exceptions will go away, which means whether they're reported to the user or not becomes irrelevant.
    I meant quite the opposite: they must be reported, though in a user-friendly way. When INSERT fails and I am notified instead that comment was posted successfully, it means that exception from database either was not propagated to UI or was swallowed and not handled properly.
  47. Bradley Schaefer said...[ Go to top ]

    Editor's note: Bradley sent this to me in email, so I'm echoing it here.
       
    I'm glad to see that these issues are finally out in the open. Having chatted with you personally it has never been 'hidden' to me, but there must be a large number of people out there who don't happen to have that sort of relationship with you for whatever reason. Also I agree with Guglielmo Lichtner who posted saying that it might be nice if you would be even MORE forthcoming in describing what the problems are. I can see how it might be embarassing to shed light on your own shortcomings, but since these problems are so evident anyway I don't think there is much to lose by being open. And you never know, you might hear some solutions in response that you would not have thought of left to your own devices. I will understand if you choose not to -- really that is what I expect -- but I figure it doesn't hurt to express my opinion. Make that post-mortem public! *grin*

    On another topic, while I am looking forward to the massive upgrade you speak of, with the launch date so far in the future I think maybe TSS' priorities should be adjusted. It gets my goat to constantly have to do battle with these problems, and asking for months of patience wears pretty thin when the problems are crippling to what is a community-participation driven site. If the community gets fed up, you may be setting yourself up to launch the new site to a far smaller audience. In fact I wonder if you might be able to compile some site statistics which would back up that hypothesis?
  48. Well, there are still a lot of issues that you don't know about. I've resisted a full postmortem because some of the problems have causes that I don't know. More than anything else, what TSS is right now is... incomplete. The conversion to Tapestry was well-known, but honestly, never *finished* - so a lot of functionality was left on the drawing board, simply because of time constraints.

    One internal fellow who's in the know said that TSS was basically a first iteration. Good start, but a lot of optimization and tuning was left out.

    Add to that a software bug here and there, and you have the current codebase... and its creakiness. It's simply unfair to point fingers, because ordinarily stuff like this would come out of a longer test cycle.

    As the original post said, these specific problems are very high priority. They will be solved, and between now and the rearchitecture, you'll have a stable TSS again.

    Now, as far as open-sourcing TSS... don't think that hasn't been discussed. In fact, a lot of things are being discussed. The biggest barrier to open-sourcing TSS is that our requirements aren't what you'd see on the surface, necessarily; an open source contributor scratches his own itch more than anything else. But don't stop thinking about that - I gots plans of my own, being an open-source author myself and a big dreamer. (Watch this space - or, rather, my own space, since TSS isn't the possession of Your Humble Author, nor should it be.)

    Going further: a discussion of the specific bugs would be fine, but some of the bugs are simply bad coding (see "time constraints" and "lack of optimization opportunity") and other bugs are simply some of those things you go through. I'm not willing to turn this into a fingerpointing contest.

    Thanks!
  49. Juozas Balioka says...[ Go to top ]

    Editor's note: Juozas emailed this to me, and I'm posting it for him.

    As I understand from stack trace it is just a connection leak ( I see DBCP error message "Timeout waiting for idle object"). Close PersistenceManager in HTTP filter or just do not pool connections (As I remember it was a PostgreSQL, it performs and scales well without any pools, this kind of application with content cache must scale on single PC without problems).
  50. Juozas Balioka says...[ Go to top ]

    Juozas, thank you. However, I can tell you that your summation of the problem is incorrect. It's not "just a connection leak." As I said in the original post and in further comments, we know what the problem is; a connection leak would be easier to manage.

    We're not using DBCP on TSS, so that assumption's easy to validate (or discard).

    It's not even a PostgreSQL scaling problem. PostgreSQL scales fine - but there's a use case (a very common use case!) on TSS that would abuse ANY database.
  51. Problems on TSS (try 4)[ Go to top ]

    In my case, after I have post a message, it doesn't appear in discussion, but I can read at the top of the right column:

    "Thank you for replying to a message on TheServerSide.com!
    Your reply has been successfully posted."

    If you manage transactions and handle exceptions properly, you will never report success after transaction fails.

    It looks like some basic application programming principles have been lost in the stack of frameworks.

    Nebojsa
  52. Problems on TSS (try 4)[ Go to top ]

    Agreed, the problem is very annoying (to say the least!) What's happening isn't frameworks, though: it's a simple bug where a notification occurs deep in the system, but... it actually doesn't occur. It gets lost. For the system, there's no exception, so there's no way for it to know that the post disappeared.

    By the way, I'm jealous. Only four tries? Yeesh, you're lucky. Imagine my problem: I have to post (HAVE to!) every day, and I can't throw my hands up in frustration like you guys can. I have to work at it until the posts appear. :(
  53. Few new features[ Go to top ]

    Talking about new features, you can take a look at any other forum of forum software and will find a lot of potential new features for TSS:

    - Post editing. Poster should be able to edit their own posts, with the indication when post was last edited. This would help non native English posters.
    - Post deletion. Posters should be able to delete their posts under certain conditions: nobody has replied to the post (or maybe nobody has viewed the post).
    - Post preview. Poster should be able to preview their posts prior to actualy posting them to the discussion.
    - Post text spell checking. All these four features would much improve discussion quality.

    Then other nice features:
    - Private messages. Postes should be able to send private messages to the other TSS members, without public audience.
    - Number of replies and views per discussion and post could be presented.

    There are a lot of usefull features in other forum software, take a look at vBulettin for example.
  54. Nebojsa Vasiljevic says...[ Go to top ]

    Editor's note: Nebojsa Vasiljevic emailed this to me, and I'm posting it for him...

    I agree with Mileta's suggestions. My additional sugeestions are:

    - Don't count news as a fist comment. Use "post comment" to comment news and "post reply" to reply on comment. In "post comment" don't offer default title. In "post reply" put default title "Re: ${originalTitle}". What you call thread is topic in a standard forum structure. I suggest structure: topic groups, topics, comments on topics and replies on comments; "news", "blogs", "discussions", ... are topic groups; each news is a topic, not a comment.

    - Show "In reply to: " line after "Post by: " in a reply to a comment.

    BTW,
    Now I get:
    "This message has already been posted once. This error results if you click on submit twice, or if you use the back button and submit the message again."
    even when I start new browser instance and go to reply.

    BTW 2,
    Re: Problems on TSS (try 4)

    I can't be sure the problem is in a framework, but the problem is in some naive design decigion.

    I have made tons of naive desingn decigions in my server-side Java projects. I suggest new major topic: "Naive design decigions". It is not "Anti patterns", it is more on "How to drive carefully with patterns, frameworsk and best practicies". My fist and biggest naive decigion is based on the opinion that J2EE is genaraly step forward from client-server.

    Nebojsa
  55. Nebojsa Vasiljevic says...[ Go to top ]

    BTW 2, Re: Problems on TSS (try 4)I can't be sure the problem is in a framework, but the problem is in some naive design decigion. I have made tons of naive desingn decigions in my server-side Java projects. I suggest new major topic: "Naive design decigions". It is not "Anti patterns", it is more on "How to drive carefully with patterns, frameworsk and best practicies". My fist and biggest naive decigion is based on the opinion that J2EE is genaraly step forward from client-server.


    Well, I have to take a bit of exception here, if you will. I don't know if it's fair to say "naive design decisions" when a full integration phase was never completed. I had a similar discussion with someone else: his response was "what idiots!" to a description of one of the problems, but that's not entirely fair.

    Why?

    Well, because during development, EVERYONE has these "duh!" moments, when their design does something incorrectly or suboptimally. In one case, I had a project that computed a cartesian product of lists that had 30000 elements in them... at incredible costs in memory and runtime. After the fact, it's easy to say "oh, my!" and criticise it, especially if it went into production - which it didn't. We cleaned it up during development.

    In this case, what you have is that "midstream" status, when something works, but not well. Criticising the developers for not finishing is all well and good, except it's not always the developers' fault. Sometimes it's just the way things are - cultural and political reasons can push things along, too.

    So "naive decisions?" Hmm, I disagree. I think "incomplete process" is more appropriate.
  56. Sory[ Go to top ]

    I have considered this part of mail private.

    Now I know that the key issue is in the process, not in the design.

    Nebojsa
  57. Sory[ Go to top ]

    I have considered this part of mail private.Now I know that the key issue is in the process, not in the design.Nebojsa

    Whoops! Sorry about that, Nebojsa - I didn't know what was private to me or meant for the TSS thread.

    attempt 0x1a
  58. Problems on TSS[ Go to top ]

    t's discouraging to see news stories on TheServerSide.com about their ongoing software problems. TSS has been a banner site for Tapestry, and chatter about problems makes for bad Tapestry press.

    The basic issue is that I signed on with The Middleware Company to do a number of phases of development for TheServerSide.com. The first phase was the basic translation of the site to a component object model, leaving all the functionality unchanged.

    At the same time this was occuring, a seperate team was converting the backend access from entity EJBs to Solarmetric Kodo.

    In the end, I had less than a week to integrate the two before going live. And yet, for the most part, the result was quite succesful -- measured partly by the amount of time that passed before anyone realized that the site had been almost completely rewritten.

    However, with the acquisition of The Middleware Company by Tech Target, my involvement with TSS came to an end; the later, more interesting phases, where we simplified the stack and built considerable UI improvements, has not come to pass. All I've seen is the introduction of more and more ads on the site.

    I can't talk to the root problem today; it is incredible frustrating that many posts get accepted and lost. That speaks to some horrible issue with transaction management and the database. Interestingly, TechTarget has not approached me for help ... this also speaks volumes as to where the problems lie.

    From observation, and from discussions with Joe Ottinger, I do know that Tapestry is doing exactly what its supposed to be doing, that the functionality problems (missing posts and such) are a problem at the application layer (the stateless session bean used to manage transactions) and the interaction between that layer, Kodo, Coherence, WebLogic and the PostgreSQL database. In fact, given the simplicity of the database schema (just six or eight tables) I suspect the problem really is in the configuration and integration of these elements.

    Based on what I've read, and some high level discussions I had with them last winter, I believe TechTarget is building a single enterprise wide solution for all their many web sites, migrating away from the Tcl-based Vignette solution used by the majority of their sites, as well as the Tapestry-based solution for TSS.com and TSS.net. All I know about the solution is that it will be based on JEE (assuming that hasn't changed since our discussions).

    #2
  59. Thanks for chiming in, Howard. I wasn't part of TSS (or TechTarget) back then, so I can't say what happened then, but I've heard that the dev cycle was, erm, incomplete - as I've suggested elsewhere in this thread.
  60. Problems on TSS[ Go to top ]

    I can confirm Howard's points that the port to the new stacks was never properly completed. TechTarget bought TSS very shortly after the initial re-launch on the new stacks, and unfortunately TSS' core developer at the time chose not to go along in the transition, and as Howard said, he was not contracted either.
                              
    So, none of this has anything to do with Tapestry or Kodo, those are both excellent tools. Although, this missed-posting problem seemed to begin after the site was very recently ported to JBoss (off of WebLogic). However, that is in no way a reflection on JBoss either, obviously the port was not properly tested.

    It's nice to see that this problem is out in the open, I hope that the comments in this thread will help raise the priority level of these problems within TechTarget.

    Floyd
  61. Problems on TSS[ Go to top ]

    0x1E
    From observation, and from discussions with Joe Ottinger, I do know that Tapestry is doing exactly what its supposed to be doing, that the functionality problems (missing posts and such) are a problem at the application layer (the stateless session bean used to manage transactions) and the interaction between that layer, Kodo, Coherence, WebLogic and the PostgreSQL database. In fact, given the simplicity of the database schema (just six or eight tables) I suspect the problem really is in the configuration and integration of these elements.

    My undertstanding is that the RMI errors are caused from the transactions timing out. That is how the current server (JBoss) reports them, but it's not a bug, that's "correct" behavior as far as I can tell. The problems are occurring at the database level itself .. seriously, the database can't even handle queries on a measely 10 concurrent connections! Patrick even volunteered his own personal time to help fix the database and the application. Unfortunately, it's not a "toy" project at all (ask Floyd!) .. TechTarget is a pretty serious business, and as such they cannot allow just anyone to play around with their production database that contains lots of valuable business data.

    p.s. if you disable a couple features that hit the DB, such as "my threads", and stop allowing traffic from spiders, the site will probably go back to "acceptable".

    Peace,

    Cameron Purdy
    Tangosol Coherence: Clustered Transactional Caching
  62. TSS.com is that important !![ Go to top ]

    What we as community hope that www.theserverside.com comes back online, fully functional (as before) and also, as early as possible without losing any of its existing features.

    The survival of (www.theserverside.com) is extremely important for the continued growth of thousands and thousands of developers/programmers.
  63. TSS.com is that important !![ Go to top ]

    The survival of (www.theserverside.com) is extremely important for the continued growth of thousands and thousands of developers/programmers.
  64. TSS.com is that important !![ Go to top ]

    The survival of (www.theserverside.com) is extremely important for the continued growth of thousands and thousands of developers/programmers.

    I totally agree with Abhay. There's no point of attacking the technologies nor the people behind it. I'm sure that if TSS felt like this entire issue(problem/solution) would be helpful for the community, they would have posted it in the first place.
    Let us just hope it gets solved quickly, so we can end this useless thread and start focussing again on real issues. This site IS important for the community, so please..
  65. feature[ Go to top ]

    use anchors to return me to the last post i read when i re-enter the thread... sometimes it takes some time to find out where you were in a big thread.
  66. Build vs Buy[ Go to top ]

    Seems to me like the real problem is trying to build your own CMS + BB software when there are scalable, stable systems already out there. I love using Java to build systems but this is not a case where I would use Java.
  67. Business you're in[ Go to top ]

    What business are you in ?
    Is the create message board systems or in providing topics
    to educate the Java community ?
    If its the later why not acquire the message board system used
    by some large sites even if its not written in Java e.g.
    VBulletin (uses PHP I believe) ?
  68. Business you're in[ Go to top ]

    protected int postAttempts=5;

    It's more than just "create message board systems." There's a lot to TSS' requirements, and besides, we have standards!

    Actually, I'm open to a lot of things. vBulletin has been suggested earlier in the thread, and I looked at it. It *is* PHP, but it's still not enough for what TSS needs, so it wouldn't fit.

    Besides, I'm a longtime TSS reader myself - and I'd be horrified if a Java Enterprise portal didn't use the technology it talked about. It'd be a bit odd to me as a user. (Wouldn't it be odd to you?)

    However, the point being that we COULD use off-the-shelf technology is a good one - and one not too far off the mark anyway. There are lots of products even in the Java space to use to fulfill parts of TSS' requirements - for example, instead of vBulletin, we might use Jive, mvnForum, SKForum or another Java forum package (this is VERY FAR from an exhaustive list).

    Again, all this is going to be addressed in the rearchitecture.
  69. Problems on TSS[ Go to top ]

    The way I see the whole thing - TSS started as one person's (ok, maybe a small training shop's) prototype project to learn J2EE (EJB mostly). As the site got popular rapidly (beyond everyone's wildest dream, which is a great thing), various commercial vendors and open source projects managed to use it as a showcase of their products. Over time the fundation became shaky, and this made Joseph Ottinger hesitates to "point fingers". Let's have the courage to say this: TSS has outgrown its original architecture and technology, and it is time for a major overhaul.
  70. Problems on TSS[ Go to top ]

    int postAttempts=2;
    The way I see the whole thing - TSS started as one person's (ok, maybe a small training shop's) prototype project to learn J2EE (EJB mostly). As the site got popular rapidly (beyond everyone's wildest dream, which is a great thing), various commercial vendors and open source projects managed to use it as a showcase of their products. Over time the fundation became shaky, and this made Joseph Ottinger hesitates to "point fingers". Let's have the courage to say this: TSS has outgrown its original architecture and technology, and it is time for a major overhaul.

    Well, as I understand it from the people involved, it wasn't meant to be a prototype project. It was written to do what it needed to do, and developed accordingly.

    TSS isn't a "showcase." The rewrite last year was a necessary update, really. (I know, some out there are going "Huh? Necessary? But now it's WORSE than it was!") Technology was chosen not for its coolness, but for how well it would fulfill the requirements. (There are some hidden requirements, as I've alluded throughout the discussion.)

    That said, I agree - major overhaul time. Either finish the original migration, or change EVERYTHING over - and, without any slight meant to the technologies used, I'd say "change everything over," because I know what some of the requirements for the rearchitecture are and a major rewrite is entirely warranted.

    As far as not pointing fingers: I wouldn't do it regardless. It's not constructive here. Nobody would learn anything they haven't already learned, and probably the most instructive thing is that tests - all kinds of tests - matter, from functional to user acceptance to performance to integration to ... you name it.
  71. guessing is fun[ Go to top ]

    It's refreshing to be in a position to take part in a guessing game and not be responsible for either fixing the problem or refuting all the guesses :-)

    My own guess is that there is primarily a complexity problem. A thick stack of frameworks that prevent people from properly understanding(?) and optimizing data access code. A forum system with an 8 table database that uses EJB and ORM?

    2
  72. int postAttempts=1;
    It's refreshing to be in a position to take part in a guessing game and not be responsible for either fixing the problem or refuting all the guesses :-)My own guess is that there is primarily a complexity problem. A thick stack of frameworks that prevent people from properly understanding(?) and optimizing data access code. A forum system with an 8 table database that uses EJB and ORM?

    BZZT! Wrong! Tommy, who's our next player? :)

    It really isn't a complexity problem (although scaling is part of the trigger for the problems, I suppose.) That's part of what makes throwing the actual issues out so hard; 99.999% of users could install the software locally, test and test and test and test, waiting for the failure.. and never see it, because they simply don't have the hardware resources to scale out enough to see the problem(s).

    (Well, in some cases, maybe not. But for one issue, this will GENERALLY be the case.)

    Besides, guys, TSS is more than just a forum, and you know it. (Or maybe you don't. Should I throw out a "what do you use at TSS?" poll sometime?)
  73. int postAttempts=1;
    It's refreshing to be in a position to take part in a guessing game and not be responsible for either fixing the problem or refuting all the guesses :-)My own guess is that there is primarily a complexity problem. A thick stack of frameworks that prevent people from properly understanding(?) and optimizing data access code. A forum system with an 8 table database that uses EJB and ORM?
    BZZT! Wrong!

    Seems my diagnose-by-ideology skills need some work then ;-)
    That's part of what makes throwing the actual issues out so hard; 99.999% of users could install the software locally, test and test and test and test, waiting for the failure.. and never see it, because they simply don't have the hardware resources to scale out enough to see the problem(s).

    Sounds really nasty. I pity the guys who have to fix this. Good luck!
  74. Actually none of this is fun[ Go to top ]

    Joseph, in all candor the biggest problem right now is the cat and mouse game you're playing right here. Either fix the problem or don't. Tell people what the problem is, or don't post. Either involve people in finding the solution or don't talk about it.

    As it is, all the hidden meanings and secret requirements and wow-keep-guessing! crap is quite tiring, as are the attempts at humor.

    It's funny for other people to say "attempt 0x1e". It's not funny at all for an editor of the site to do it.

    I noticed TSS was having increasing problems in the 1st week of September (actually, it may go back into August). October is now around the corner. Just fix the damn thing, or watch the community evaporate.
  75. Actually none of this is fun[ Go to top ]

    Joseph, in all candor the biggest problem right now is the cat and mouse game you're playing right here. Either fix the problem or don't. Tell people what the problem is, or don't post. Either involve people in finding the solution or don't talk about it.As it is, all the hidden meanings and secret requirements and wow-keep-guessing! crap is quite tiring, as are the attempts at humor...

    I cannot agree more. I've been reading TSS since the very first day and seeing the current state of TSS is quite sad. Enough already, just tell people what the problem is or just shut the site down (or go back to weblogic until you can fix it on jboss). Good grief!

    -Duc Nguyen
  76. Actually none of this is fun[ Go to top ]

    I cannot agree more. I've been reading TSS since the very first day and seeing the current state of TSS is quite sad. Enough already, just tell people what the problem is or just shut the site down (or go back to weblogic until you can fix it on jboss). Good grief!-Duc Nguyen

    +1
  77. Actually none of this is fun - agreed[ Go to top ]

    Guys, I'm not trying to play a "cat and mouse" game. I'm not trying to invite you to figure out what the problem is. That part's been done.

    As far as fixes are concerned, they're coming.
  78. News in "Aftonbladet"
     today:
    Great computer meltdown at Föreningsbanken

    2 mill customers is affected, they can not draw money from ATM or use the internet services or shop for larger amounts. One person for example had to leave all wares he had bought in the Supermarket because he was not allowed to pay with the card or to draw money in the automat outside the store.

    Sun Case Study:
    http://www.sun.com/service/sunps/success/case_studies/swedbank_cs2.html

    (Sorry this is just an guest appearance, will not be repeated :)

    best regards
    Rolf Tollerud