[#43924] - [cypress] PostgreSQL Connection pool for System Tests
- Fixed in Code Base
- 29 Aug 2024
- Medium
- Build: 4.4-dev
- # 43924
- Diff
- muhme:fix-postgres-sessions
- Pending continuous-integration/drone/pr Build is pending Details
User tests: Successful: Unsuccessful:
Summary of Changes
Use Connection pool for PostgreSQL in System Tests.
Sometimes single specs in the System Tests fail with:
CypressError: `cy.task('queryDB')` failed with the following error:
> duplicate key value violates unique constraint "cpostgresmax_users_pkey"
The error occurs on various specs in the System Tests, especially when running on drone.
It could be reproduced locally in running the System Tests tree times (first with installation, 2nd and 3rd run w/o):
npx cypress run
npx cypress run --spec 'tests/System/integration/administrator/**/*.cy.{js,jsx,ts,tsx},tests/System/integration/site/**/*.cy.{js,jsx,ts,tsx},tests/System/integration/api/**/*.cy.{js,jsx,ts,tsx},tests/System/integration/plugins/**/*.cy.{js,jsx,ts,tsx}'
npx cypress run --spec 'tests/System/integration/administrator/**/*.cy.{js,jsx,ts,tsx},tests/System/integration/site/**/*.cy.{js,jsx,ts,tsx},tests/System/integration/api/**/*.cy.{js,jsx,ts,tsx},tests/System/integration/plugins/**/*.cy.{js,jsx,ts,tsx}'
Checking the session usage in pgAdmin shows the maximum number of 100 (SHOW max_connections) configured sessions appears to have been reached:
Therefore the usage of postgres connection pool is implemented with max 10 connections.
The session usage is reduced to max 12 and the error could no more reproduced:
Testing Instructions
Having Joomla instance running with PostgreSQL (PDO). Running System Tests and check Sessions tab in pgAdmin.
Actual result BEFORE applying this Pull Request
Heavy session usage during the System Tests. System Tests have sometimes the error duplicate key value violates unique constraint "cpostgresmax_users_pkey".
Expected result AFTER applying this Pull Request
Maximum number of 12 sessions used during the System Tests. System Tests running with all specs passed.
Link to documentations
Please select:
-
Documentation link for docs.joomla.org:
-
No documentation changes for docs.joomla.org needed
-
Pull Request link for manual.joomla.org:
-
No documentation changes for manual.joomla.org needed
joomla-cms-bot
- | Category | ⇒ | JavaScript Unit Tests |
| Status | New | ⇒ | Pending |
| Labels |
Added:
Unit/System Tests
PR-4.4-dev
|
||
This PR was successfully tested with pgsql in the current branches 4.4-dev, 5.1-dev, 5.2-dev, 5.3-dev and 6.0-dev.
Despite the positive tests and good looking charts, the original bug was seen once with 5.2-dev on Apple M3, even though this PR was installed:
Running: administrator/components/com_privacy/Consent.cy.js (28 of 123)
Test in backend that privacy consent component
✓ can view privacy consents (391ms)
✓ can invalidate privacy consent (538ms)
✓ can invalidate all privacy consents (548ms)
✓ can filter by valid consent (545ms)
✓ can filter by invalidated consent (552ms)
✓ can filter by obsolete consent (537ms)
✓ can search username (1109ms)
✓ can search user id (1108ms)
✓ can search consent id (1040ms)
✓ can display correct number of consents (7874ms)
1) can list by status in ascending order
2) "after each" hook for "can list by status in ascending order"
10 passing (16s)
2 failing
1) Test in backend that privacy consent component
can list by status in ascending order:
CypressError: `cy.task('queryDB')` failed with the following error:
> duplicate key value violates unique constraint "jos52_users_pkey"
In my opinion, this PR is still useful. Without it, it doesn't work at all on an Apple M3 - you get too many clients. But the duplicate key bug needs further investigation (The first interesting detail is that this time it is a different constraint).
But, the problem could not be reproduced in 3 complete runs 5.2-dev and also not with 3 further tests administrator/components/com_privacy/Consent.cy.js. It also did not occur in the other four active branches.
| Status | Pending | ⇒ | Fixed in Code Base |
| Closed_Date | 0000-00-00 00:00:00 | ⇒ | 2024-08-29 09:36:41 |
| Closed_By | ⇒ | laoneo |
Thanks!
? And, definitely it fixes PostgresError: sorry, too many clients already as I see it regulary on M3, it may reduce duplicate key value, definitely do not fix duplicate key value - I am still searching ...
I am now happy to have a faster Apple M3 for testing, the 4.4-dev system tests (without installation step) run in only 3:10 with mariadbi. And no wonder that the pgsql error appears here too. But this time with a clear error message:
With this PR the system tests are passed in 3:07 for pgsql ?