Alfresco, along with
Red Hat and
MySQL, has published a white paper covering the creation of a benchmark for JCR (a topic often-discussed on the Apache Jackrabbit lists.) The paper (titled "
The Fastest JCR Repository," freely downloadable but requires registration) addresses Alfresco's JCR implementation, and could be broader in application, but they're also going to make the benchmark open source. The Alfresco numbers could serve as a reference point.
A true performance profile, not just for Alfresco, but for the stack being used can be determined to deliver a reflective cost in terms of dollars per transaction per second. Benchmark metrics are assessed by independent experts through a formal auditing process so that they assure compliance with pre-determined benchmark rules. This validation process gives customer’s confidence when comparing the performance of different stack configurations.
The benchmark is based on the following phases and principles:- Uses the JSR-170 Industry standard API
- Large repository with 10 million documents and above across 10,000 folders
And the following phases- Perform a continuous load of documents, that is transactionally safe, to demonstrate linear scalability for document uploads, as the size of the repository increases
- The document load is performed with a failure, to validate that transactionality is maintained and the repository is not corrupted
- Mixed, concurrent read and write to demonstrate linear sub-second response times without performance degradation due to concurrency issues
With- Different User Loads
- Different Machine Configurations
The white paper then walks through the benchmark process, showing what elements were tested, and what requirements were fulfilled. It then produces two sets of results, based on the ratio of reads and writes for the repository.
The mixed read/write benchmark delivered the following performance results:- Read Content – 0.34961s
- Read Property - 0.41976s
- Create Content – 0.58788s
- Create Folder – 0.54419
The paper doesn't compare other stacks, so these numbers may be fantastic - or they might not. When the benchmark is made available, the Alfresco numbers will be able to be compared and the test validated.
Regardless of the benchmark scores, the fact that Alfresco is committing resources to this effort is excellent news.