topaLE
/
vaultwarden
mirror of https://github.com/dani-garcia/vaultwarden
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
Browse Source

Add playwright tests

pull/3899/head
Timshel 7 months ago
parent
524d65e772
commit
927968d251
31 changed files with 4269 additions and 3 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 7
      SSO.md
  2. 63
      playwright/.env.template
  3. 6
      playwright/.gitignore
  4. 177
      playwright/README.md
  5. 40
      playwright/compose/keycloak/Dockerfile
  6. 36
      playwright/compose/keycloak/setup.sh
  7. 40
      playwright/compose/playwright/Dockerfile
  8. 39
      playwright/compose/vaultwarden/Dockerfile
  9. 24
      playwright/compose/vaultwarden/build.sh
  10. 121
      playwright/docker-compose.yml
  11. 22
      playwright/global-setup.ts
  12. 219
      playwright/global-utils.ts
  13. 2364
      playwright/package-lock.json
  14. 21
      playwright/package.json
  15. 137
      playwright/playwright.config.ts
  16. 90
      playwright/test.env
  17. 163
      playwright/tests/login.smtp.spec.ts
  18. 94
      playwright/tests/login.spec.ts
  19. 164
      playwright/tests/organization.spec.ts
  20. 7
      playwright/tests/setups/db-setup.ts
  21. 11
      playwright/tests/setups/db-teardown.ts
  22. 9
      playwright/tests/setups/db-test.ts
  23. 19
      playwright/tests/setups/sso-setup.ts
  24. 15
      playwright/tests/setups/sso-teardown.ts
  25. 111
      playwright/tests/setups/sso.ts
  26. 47
      playwright/tests/setups/user.ts
  27. 78
      playwright/tests/sso_login.spec.ts
  28. 142
      playwright/tests/sso_organization.spec.ts
  29. 2
      src/static/templates/email/send_org_invite.html.hbs
  30. 2
      src/static/templates/email/twofactor_email.html.hbs
  31. 2
      src/static/templates/email/verify_email.html.hbs

7
SSO.md
View File

@ -99,6 +99,13 @@ Server configuration, nothing specific just set:
- `SSO_CLIENT_SECRET`
- `SSO_PKCE=true`
### Testing
If you want to run a testing instance of Keycloak the Playwright [docker-compose](playwright/docker-compose.yml) can be used.
\
More details on how to use it in [README.md](playwright/README.md#openid-connect-test-setup).
## Auth0
Not working due to the following issue https://github.com/ramosbugs/openidconnect-rs/issues/23 (they appear not to follow the spec).

63
playwright/.env.template
View File

@ -0,0 +1,63 @@
#################################
### Conf to run dev instances ###
#################################
ENV=dev
DC_ENV_FILE=.env
COMPOSE_IGNORE_ORPHANS=True
DOCKER_BUILDKIT=1
################
# Users Config #
################
TEST_USER=test
TEST_USER_PASSWORD=${TEST_USER}
TEST_USER_MAIL=${TEST_USER}@yopmail.com
TEST_USER2=test2
TEST_USER2_PASSWORD=${TEST_USER2}
TEST_USER2_MAIL=${TEST_USER2}@yopmail.com
TEST_USER3=test3
TEST_USER3_PASSWORD=${TEST_USER3}
TEST_USER3_MAIL=${TEST_USER3}@yopmail.com
###################
# Keycloak Config #
###################
KEYCLOAK_ADMIN=admin
KEYCLOAK_ADMIN_PASSWORD=${KEYCLOAK_ADMIN}
KC_HTTP_HOST=127.0.0.1
KC_HTTP_PORT=8080
# Script parameters (use Keycloak and VaultWarden config too)
TEST_REALM=test
DUMMY_REALM=dummy
DUMMY_AUTHORITY=http://${KC_HTTP_HOST}:${KC_HTTP_PORT}/realms/${DUMMY_REALM}
######################
# Vaultwarden Config #
######################
ROCKET_ADDRESS=0.0.0.0
ROCKET_PORT=8000
DOMAIN=http://127.0.0.1:${ROCKET_PORT}
I_REALLY_WANT_VOLATILE_STORAGE=true
SSO_ENABLED=true
SSO_ONLY=false
SSO_CLIENT_ID=VaultWarden
SSO_CLIENT_SECRET=VaultWarden
SSO_AUTHORITY=http://${KC_HTTP_HOST}:${KC_HTTP_PORT}/realms/${TEST_REALM}
SMTP_HOST=127.0.0.1
SMTP_PORT=1025
SMTP_SECURITY=off
SMTP_TIMEOUT=5
SMTP_FROM=vaultwarden@test
SMTP_FROM_NAME=Vaultwarden
########################################################
# DUMMY values for docker-compose to stop bothering us #
########################################################
MARIADB_PORT=3305
MYSQL_PORT=3307
POSTGRES_PORT=5432

6
playwright/.gitignore
View File

@ -0,0 +1,6 @@
logs
node_modules/
/test-results/
/playwright-report/
/playwright/.cache/
temp

177
playwright/README.md
View File

@ -0,0 +1,177 @@
# Integration tests
This allows running integration tests using [Playwright](https://playwright.dev/).
\
It usse its own [test.env](/test/scenarios/test.env) with different ports to not collide with a running dev instance.
## Install
This rely on `docker` and the `compose` [plugin](https://docs.docker.com/compose/install/).
Databases (`Mariadb`, `Mysql` and `Postgres`) and `Playwright` will run in containers.
### Running Playwright outside docker
It's possible to run `Playwright` outside of the container, this remove the need to rebuild the image for each change.
You'll additionally need `nodejs` then run:
```bash
npm install
npx playwright install-deps
npx playwright install firefox
```
## Usage
To run all the tests:
```bash
DOCKER_BUILDKIT=1 docker compose --profile playwright --env-file test.env run Playwright
```
To force a rebuild of the Playwright image:
```bash
DOCKER_BUILDKIT=1 docker compose --env-file test.env build Playwright
```
To access the ui to easily run test individually and debug if needed (will not work in docker):
```bash
npx playwright test --ui
```
### DB
Projects are configured to allow to run tests only on specific database.
\
You can use:
```bash
DOCKER_BUILDKIT=1 docker compose --profile playwright --env-file test.env run Playwright test --project=mariadb
DOCKER_BUILDKIT=1 docker compose --profile playwright --env-file test.env run Playwright test --project=mysql
DOCKER_BUILDKIT=1 docker compose --profile playwright --env-file test.env run Playwright test --project=postgres
DOCKER_BUILDKIT=1 docker compose --profile playwright --env-file test.env run Playwright test --project=sqlite
```
### SSO
To run the SSO tests:
```bash
DOCKER_BUILDKIT=1 docker compose --profile playwright --env-file test.env run Playwright test --project sso-sqlite
```
### Keep services running
If you want you can keep the Db and Keycloak runnning (states are not impacted by the tests):
```bash
PW_KEEP_SERVICE_RUNNNING=true npx playwright test
```
### Running specific tests
To run a whole file you can :
```bash
DOCKER_BUILDKIT=1 docker compose --profile playwright --env-file test.env run Playwright test --project=sqlite tests/login.spec.ts
DOCKER_BUILDKIT=1 docker compose --profile playwright --env-file test.env run Playwright test --project=sqlite login
```
To run only a specifc test (It might fail if it has dependency):
```bash
DOCKER_BUILDKIT=1 docker compose --profile playwright --env-file test.env run Playwright test --project=sqlite -g "Account creation"
DOCKER_BUILDKIT=1 docker compose --profile playwright --env-file test.env run Playwright test --project=sqlite tests/login.spec.ts:16
```
## Writing scenario
When creating new scenario use the recorder to more easily identify elements (in general try to rely on visible hint to identify elements and not hidden ids).
This does not start the server, you will need to start it manually.
```bash
npx playwright codegen "http://127.0.0.1:8000"
```
## Override web-vault
It's possible to change the `web-vault` used by referencing a different `bw_web_builds` commit.
```bash
export PW_WV_REPO_URL=https://github.com/Timshel/oidc_web_builds.git
export PW_WV_COMMIT_HASH=8707dc76df3f0cceef2be5bfae37bb29bd17fae6
DOCKER_BUILDKIT=1 docker compose --profile playwright --env-file test.env build Playwright
```
# OpenID Connect test setup
Additionnaly this `docker-compose` template allow to run locally `VaultWarden`, [Keycloak](https://www.keycloak.org/) and [Maildev](https://github.com/timshel/maildev) to test OIDC.
## Setup
This rely on `docker` and the `compose` [plugin](https://docs.docker.com/compose/install/).
First create a copy of `.env.template` as `.env` (This is done to prevent commiting your custom settings, Ex `SMTP_`).
## Usage
Then start the stack (the `profile` is required to run `Vaultwarden`) :
```bash
> docker compose --profile vaultwarden --env-file .env up
....
keycloakSetup_1 | Logging into http://127.0.0.1:8080 as user admin of realm master
keycloakSetup_1 | Created new realm with id 'test'
keycloakSetup_1 | 74af4933-e386-4e64-ba15-a7b61212c45e
oidc_keycloakSetup_1 exited with code 0
```
Wait until `oidc_keycloakSetup_1 exited with code 0` which indicate the correct setup of the Keycloak realm, client and user (It's normal for this container to stop once the configuration is done).
Then you can access :
- `VaultWarden` on http://0.0.0.0:8000 with the default user `test@yopmail.com/test`.
- `Keycloak` on http://0.0.0.0:8080/admin/master/console/ with the default user `admin/admin`
- `Maildev` on http://0.0.0.0:1080
To proceed with an SSO login after you enter the email, on the screen prompting for `Master Password` the SSO button should be visible.
To use your computer external ip (for example when testing with a phone) you will have to configure `KC_HTTP_HOST` and `DOMAIN`.
## Running only Keycloak
You can run just `Keycloak` with `--profile keycloak`:
```bash
> docker compose --profile keycloak --env-file .env up
```
When running with a local VaultWarden and the default `web-vault` you'll need to make the SSO button visible using :
```bash
sed -i 's#a\[routerlink="/sso"\],##' web-vault/app/main.*.css
```
Otherwise you'll need to reveal the SSO login button using the debug console (F12)
```js
document.querySelector('a[routerlink="/sso"]').style.setProperty("display", "inline-block", "important");
```
## Rebuilding the Vaultwarden
To force rebuilding the Vaultwarden image you can run
```bash
docker compose --profile vaultwarden --env-file .env build VaultwardenPrebuild Vaultwarden
```
## Configuration
All configuration for `keycloak` / `VaultWarden` / `keycloak_setup.sh` can be found in [.env](.env.template).
The content of the file will be loaded as environment variables in all containers.
- `keycloak` [configuration](https://www.keycloak.org/server/all-config) include `KEYCLOAK_ADMIN` / `KEYCLOAK_ADMIN_PASSWORD` and any variable prefixed `KC_` ([more information](https://www.keycloak.org/server/configuration#_example_configuring_the_db_url_host_parameter)).
- All `VaultWarden` configuration can be set (EX: `SMTP_*`)
## Cleanup
Use `docker compose --profile vaultWarden down`.

40
playwright/compose/keycloak/Dockerfile
View File

@ -0,0 +1,40 @@
FROM docker.io/library/debian:bookworm-slim as build
ENV DEBIAN_FRONTEND=noninteractive
ARG KEYCLOAK_VERSION
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
RUN apt-get update \
&& apt-get install -y ca-certificates curl wget \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /
RUN wget -c https://github.com/keycloak/keycloak/releases/download/${KEYCLOAK_VERSION}/keycloak-${KEYCLOAK_VERSION}.tar.gz -O - | tar -xz
FROM docker.io/library/debian:bookworm-slim
ENV DEBIAN_FRONTEND=noninteractive
ARG KEYCLOAK_VERSION
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
RUN apt-get update \
&& apt-get install -y ca-certificates curl wget \
&& rm -rf /var/lib/apt/lists/*
ARG JAVA_URL
ARG JAVA_VERSION
ENV JAVA_VERSION=${JAVA_VERSION}
RUN mkdir -p /opt/openjdk && cd /opt/openjdk \
&& wget -c "${JAVA_URL}" -O - | tar -xz
WORKDIR /
COPY setup.sh /setup.sh
COPY --from=build /keycloak-${KEYCLOAK_VERSION}/bin /opt/keycloak/bin
CMD "/setup.sh"

36
playwright/compose/keycloak/setup.sh
View File

@ -0,0 +1,36 @@
#!/bin/bash
export PATH=/opt/keycloak/bin:/opt/openjdk/jdk-${JAVA_VERSION}/bin:$PATH
export JAVA_HOME=/opt/openjdk/jdk-${JAVA_VERSION}
STATUS_CODE=0
while [[ "$STATUS_CODE" != "404" ]] ; do
echo "Will retry in 2 seconds"
sleep 2
STATUS_CODE=$(curl -s -o /dev/null -w "%{http_code}" "$DUMMY_AUTHORITY")
if [[ "$STATUS_CODE" = "200" ]]; then
echo "Setup should already be done. Will not run."
exit 0
fi
done
set -e
kcadm.sh config credentials --server "http://${KC_HTTP_HOST}:${KC_HTTP_PORT}" --realm master --user "$KEYCLOAK_ADMIN" --password "$KEYCLOAK_ADMIN_PASSWORD" --client admin-cli
kcadm.sh create realms -s realm="$TEST_REALM" -s enabled=true -s "accessTokenLifespan=600"
kcadm.sh create clients -r test -s "clientId=$SSO_CLIENT_ID" -s "secret=$SSO_CLIENT_SECRET" -s "redirectUris=[\"$DOMAIN/*\"]" -i
TEST_USER_ID=$(kcadm.sh create users -r "$TEST_REALM" -s "username=$TEST_USER" -s "firstName=$TEST_USER" -s "lastName=$TEST_USER" -s "email=$TEST_USER_MAIL" -s emailVerified=true -s enabled=true -i)
kcadm.sh update users/$TEST_USER_ID/reset-password -r "$TEST_REALM" -s type=password -s "value=$TEST_USER_PASSWORD" -n
TEST_USER2_ID=$(kcadm.sh create users -r "$TEST_REALM" -s "username=$TEST_USER2" -s "firstName=$TEST_USER2" -s "lastName=$TEST_USER2" -s "email=$TEST_USER2_MAIL" -s emailVerified=true -s enabled=true -i)
kcadm.sh update users/$TEST_USER2_ID/reset-password -r "$TEST_REALM" -s type=password -s "value=$TEST_USER2_PASSWORD" -n
TEST_USER3_ID=$(kcadm.sh create users -r "$TEST_REALM" -s "username=$TEST_USER3" -s "firstName=$TEST_USER3" -s "lastName=$TEST_USER3" -s "email=$TEST_USER3_MAIL" -s emailVerified=true -s enabled=true -i)
kcadm.sh update users/$TEST_USER3_ID/reset-password -r "$TEST_REALM" -s type=password -s "value=$TEST_USER3_PASSWORD" -n
# Dummy realm to mark end of setup
kcadm.sh create realms -s realm="$DUMMY_REALM" -s enabled=true -s "accessTokenLifespan=600"

40
playwright/compose/playwright/Dockerfile
View File

@ -0,0 +1,40 @@
FROM docker.io/library/debian:bookworm-slim
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update \
&& apt-get install -y ca-certificates curl \
&& curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc \
&& chmod a+r /etc/apt/keyrings/docker.asc \
&& echo "deb [signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian bookworm stable" | tee /etc/apt/sources.list.d/docker.list \
&& apt-get update \
&& apt-get install -y --no-install-recommends \
containerd.io \
docker-buildx-plugin \
docker-ce \
docker-ce-cli \
docker-compose-plugin \
git \
libmariadb-dev-compat \
libpq5 \
nodejs \
npm \
openssl \
&& rm -rf /var/lib/apt/lists/*
RUN mkdir /playwright
WORKDIR /playwright
COPY package.json .
RUN npm install && npx playwright install-deps && npx playwright install firefox
COPY docker-compose.yml test.env ./
COPY compose ./compose
COPY *.ts test.env ./
COPY tests ./tests
ENTRYPOINT ["/usr/bin/npx", "playwright"]
CMD ["test"]

39
playwright/compose/vaultwarden/Dockerfile
View File

@ -0,0 +1,39 @@
FROM playwright_oidc_vaultwarden_prebuilt AS vaultwarden
FROM node:18-bookworm AS build
arg REPO_URL
arg COMMIT_HASH
ENV REPO_URL=$REPO_URL
ENV COMMIT_HASH=$COMMIT_HASH
COPY --from=vaultwarden /web-vault /web-vault
COPY build.sh /build.sh
RUN /build.sh
######################## RUNTIME IMAGE ########################
FROM docker.io/library/debian:bookworm-slim
ENV DEBIAN_FRONTEND=noninteractive
# Create data folder and Install needed libraries
RUN mkdir /data && \
apt-get update && apt-get install -y \
--no-install-recommends \
ca-certificates \
curl \
libmariadb-dev-compat \
libpq5 \
openssl && \
rm -rf /var/lib/apt/lists/*
# Copies the files from the context (Rocket.toml file and web-vault)
# and the binary from the "build" stage to the current stage
WORKDIR /
COPY --from=vaultwarden /start.sh .
COPY --from=vaultwarden /vaultwarden .
COPY --from=build /web-vault ./web-vault
ENTRYPOINT ["/start.sh"]

24
playwright/compose/vaultwarden/build.sh
View File

@ -0,0 +1,24 @@
#!/bin/bash
echo $REPO_URL
echo $COMMIT_HASH
if [[ ! -z "$REPO_URL" ]] && [[ ! -z "$COMMIT_HASH" ]] ; then
rm -rf /web-vault
mkdir bw_web_builds;
cd bw_web_builds;
git -c init.defaultBranch=main init
git remote add origin "$REPO_URL"
git fetch --depth 1 origin "$COMMIT_HASH"
git -c advice.detachedHead=false checkout FETCH_HEAD
export VAULT_VERSION=$(cat Dockerfile | grep "ARG VAULT_VERSION" | cut -d "=" -f2)
./scripts/checkout_web_vault.sh
./scripts/patch_web_vault.sh
./scripts/build_web_vault.sh
printf '{"version":"%s"}' "$COMMIT_HASH" > ./web-vault/apps/web/build/vw-version.json
mv ./web-vault/apps/web/build /web-vault
fi

121
playwright/docker-compose.yml
View File

@ -0,0 +1,121 @@
services:
VaultwardenPrebuild:
profiles: ["playwright", "vaultwarden"]
container_name: playwright_oidc_vaultwarden_prebuilt
image: playwright_oidc_vaultwarden_prebuilt
build:
context: ..
dockerfile: Dockerfile
entrypoint: /bin/bash
restart: "no"
Vaultwarden:
profiles: ["playwright", "vaultwarden"]
container_name: playwright_oidc_vaultwarden-${ENV:-dev}
image: playwright_oidc_vaultwarden-${ENV:-dev}
network_mode: "host"
build:
context: compose/vaultwarden
dockerfile: Dockerfile
args:
REPO_URL: ${PW_WV_REPO_URL:-}
COMMIT_HASH: ${PW_WV_COMMIT_HASH:-}
env_file: ${DC_ENV_FILE:-.env}
environment:
- DATABASE_URL
- I_REALLY_WANT_VOLATILE_STORAGE
- SMTP_HOST
- SMTP_FROM
- SMTP_DEBUG
- SSO_FRONTEND
- SSO_ENABLED
- SSO_ONLY
restart: "no"
depends_on:
- VaultwardenPrebuild
Playwright:
profiles: ["playwright"]
container_name: playwright_oidc_playwright
image: playwright_oidc_playwright
network_mode: "host"
build:
context: .
dockerfile: compose/playwright/Dockerfile
environment:
- PW_WV_REPO_URL
- PW_WV_COMMIT_HASH
restart: "no"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ..:/project
Mariadb:
profiles: ["playwright"]
container_name: playwright_mariadb
image: mariadb:11.2.4
env_file: test.env
healthcheck:
test: ["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"]
start_period: 10s
interval: 10s
ports:
- ${MARIADB_PORT}:3306
Mysql:
profiles: ["playwright"]
container_name: playwright_mysql
image: mysql:8.4.1
env_file: test.env
healthcheck:
test: ["CMD", "mysqladmin" ,"ping", "-h", "localhost"]
start_period: 10s
interval: 10s
ports:
- ${MYSQL_PORT}:3306
Postgres:
profiles: ["playwright"]
container_name: playwright_postgres
image: postgres:16.3
env_file: test.env
healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
start_period: 20s
interval: 30s
ports:
- ${POSTGRES_PORT}:5432
Maildev:
profiles: ["vaultwarden", "maildev"]
container_name: maildev
image: timshel/maildev
ports:
- ${SMTP_PORT}:1025
- 1080:1080
Keycloak:
profiles: ["keycloak", "vaultwarden"]
container_name: keycloak-${ENV:-dev}
image: quay.io/keycloak/keycloak:25.0.4
network_mode: "host"
command:
- start-dev
env_file: ${DC_ENV_FILE:-.env}
KeycloakSetup:
profiles: ["keycloak", "vaultwarden"]
container_name: keycloakSetup-${ENV:-dev}
image: keycloak_setup-${ENV:-dev}
build:
context: compose/keycloak
dockerfile: Dockerfile
args:
KEYCLOAK_VERSION: 25.0.4
JAVA_URL: https://download.java.net/java/GA/jdk21.0.2/f2283984656d49d69e91c558476027ac/13/GPL/openjdk-21.0.2_linux-x64_bin.tar.gz
JAVA_VERSION: 21.0.2
network_mode: "host"
depends_on:
- Keycloak
restart: "no"
env_file: ${DC_ENV_FILE:-.env}

22
playwright/global-setup.ts
View File

@ -0,0 +1,22 @@
import { firefox, type FullConfig } from '@playwright/test';
import { execSync } from 'node:child_process';
import fs from 'fs';
const utils = require('./global-utils');
utils.loadEnv();
async function globalSetup(config: FullConfig) {
// Are we running in docker and the project is mounted ?
const path = (fs.existsSync("/project/playwright/playwright.config.ts") ? "/project/playwright" : ".");
execSync(`docker compose --project-directory ${path} --profile playwright --env-file test.env build VaultwardenPrebuild`, {
env: { ...process.env },
stdio: "inherit"
});
execSync(`docker compose --project-directory ${path} --profile playwright --env-file test.env build Vaultwarden`, {
env: { ...process.env },
stdio: "inherit"
});
}
export default globalSetup;

219
playwright/global-utils.ts
View File

@ -0,0 +1,219 @@
import { type Browser, type TestInfo } from '@playwright/test';
import { EventEmitter } from "events";
import { type Mail, MailServer } from 'maildev';
import { execSync } from 'node:child_process';
import dotenv from 'dotenv';
import dotenvExpand from 'dotenv-expand';
const fs = require("fs");
const { spawn } = require('node:child_process');
export function loadEnv(){
var myEnv = dotenv.config({ path: 'test.env' });
dotenvExpand.expand(myEnv);
return {
user1: {
email: process.env.TEST_USER_MAIL,
name: process.env.TEST_USER,
password: process.env.TEST_USER_PASSWORD,
},
user2: {
email: process.env.TEST_USER2_MAIL,
name: process.env.TEST_USER2,
password: process.env.TEST_USER2_PASSWORD,
},
user3: {
email: process.env.TEST_USER3_MAIL,
name: process.env.TEST_USER3,
password: process.env.TEST_USER3_PASSWORD,
},
}
}
export function closeMails(mailServer: MailServer, mailIterators: AsyncIterator<Mail>[]) {
if( mailServer ) {
mailServer.close();
}
if( mailIterators ) {
for (const mails of mailIterators) {
if(mails){
mails.return();
}
}
}
}
export async function waitFor(url: String, browser: Browser) {
var ready = false;
var context;
do {
try {
context = await browser.newContext();
const page = await context.newPage();
await page.waitForTimeout(500);
const result = await page.goto(url);
ready = result.status() === 200;
} catch(e) {
if( !e.message.includes("CONNECTION_REFUSED") ){
throw e;
}
} finally {
await context.close();
}
} while(!ready);
}
export function startComposeService(serviceName: String){
console.log(`Starting ${serviceName}`);
execSync(`docker compose --profile playwright --env-file test.env up -d ${serviceName}`);
}
export function stopComposeService(serviceName: String){
console.log(`Stopping ${serviceName}`);
execSync(`docker compose --profile playwright --env-file test.env stop ${serviceName}`);
}
function wipeSqlite(){
console.log(`Delete Vaultwarden container to wipe sqlite`);
execSync(`docker compose --env-file test.env stop Vaultwarden`);
execSync(`docker compose --env-file test.env rm -f Vaultwarden`);
}
async function wipeMariaDB(){
var mysql = require('mysql2/promise');
var ready = false;
var connection;
do {
try {
connection = await mysql.createConnection({
user: process.env.MARIADB_USER,
host: "127.0.0.1",
database: process.env.MARIADB_DATABASE,
password: process.env.MARIADB_PASSWORD,
port: process.env.MARIADB_PORT,
});
await connection.execute(`DROP DATABASE ${process.env.MARIADB_DATABASE}`);
await connection.execute(`CREATE DATABASE ${process.env.MARIADB_DATABASE}`);
console.log('Successfully wiped mariadb');
ready = true;
} catch (err) {
console.log(`Error when wiping mariadb: ${err}`);
} finally {
if( connection ){
connection.end();
}
}
await new Promise(r => setTimeout(r, 1000));
} while(!ready);
}
async function wipeMysqlDB(){
var mysql = require('mysql2/promise');
var ready = false;
var connection;
do{
try {
connection = await mysql.createConnection({
user: process.env.MYSQL_USER,
host: "127.0.0.1",
database: process.env.MYSQL_DATABASE,
password: process.env.MYSQL_PASSWORD,
port: process.env.MYSQL_PORT,
});
await connection.execute(`DROP DATABASE ${process.env.MYSQL_DATABASE}`);
await connection.execute(`CREATE DATABASE ${process.env.MYSQL_DATABASE}`);
console.log('Successfully wiped mysql');
ready = true;
} catch (err) {
console.log(`Error when wiping mysql: ${err}`);
} finally {
if( connection ){
connection.end();
}
}
await new Promise(r => setTimeout(r, 1000));
} while(!ready);
}
async function wipePostgres(){
const { Client } = require('pg');
const client = new Client({
user: process.env.POSTGRES_USER,
host: "127.0.0.1",
database: "postgres",
password: process.env.POSTGRES_PASSWORD,
port: process.env.POSTGRES_PORT,
});
try {
await client.connect();
await client.query(`DROP DATABASE ${process.env.POSTGRES_DB}`);
await client.query(`CREATE DATABASE ${process.env.POSTGRES_DB}`);
console.log('Successfully wiped postgres');
} catch (err) {
console.log(`Error when wiping postgres: ${err}`);
} finally {
client.end();
}
}
function dbConfig(testInfo: TestInfo){
switch(testInfo.project.name) {
case "postgres": return {
DATABASE_URL: `postgresql://${process.env.POSTGRES_USER}:${process.env.POSTGRES_PASSWORD}@127.0.0.1:${process.env.POSTGRES_PORT}/${process.env.POSTGRES_DB}`
}
case "mariadb": return {
DATABASE_URL: `mysql://${process.env.MARIADB_USER}:${process.env.MARIADB_PASSWORD}@127.0.0.1:${process.env.MARIADB_PORT}/${process.env.MARIADB_DATABASE}`
}
case "mysql": return {
DATABASE_URL: `mysql://${process.env.MYSQL_USER}:${process.env.MYSQL_PASSWORD}@127.0.0.1:${process.env.MYSQL_PORT}/${process.env.MYSQL_DATABASE}`
}
default: return { I_REALLY_WANT_VOLATILE_STORAGE: true }
}
}
/**
* All parameters passed in `env` need to be added to the docker-compose.yml
**/
export async function startVaultwarden(browser: Browser, testInfo: TestInfo, env = {}, resetDB: Boolean = true) {
if( resetDB ){
switch(testInfo.project.name) {
case "postgres":
await wipePostgres();
break;
case "mariadb":
await wipeMariaDB();
break;
case "mysql":
await wipeMysqlDB();
break;
default:
wipeSqlite();
}
}
console.log(`Starting Vaultwarden`);
execSync(`docker compose --profile playwright --env-file test.env up -d Vaultwarden`, {
env: { ...env, ...dbConfig(testInfo) },
});
await waitFor("/", browser);
console.log(`Vaultwarden running on: ${process.env.DOMAIN}`);
}
export async function stopVaultwarden() {
console.log(`Vaultwarden stopping`);
execSync(`docker compose --profile playwright --env-file test.env stop Vaultwarden`);
}
export async function restartVaultwarden(page: Page, testInfo: TestInfo, env, resetDB: Boolean = true) {
stopVaultwarden();
return startVaultwarden(page.context().browser(), testInfo, env, resetDB);
}

2364
playwright/package-lock.json
View File

File diff suppressed because it is too large

21
playwright/package.json
View File

@ -0,0 +1,21 @@
{
"name": "scenarios",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {},
"keywords": [],
"author": "",
"license": "ISC",
"devDependencies": {
"@playwright/test": "^1.49.1",
"dotenv": "^16.4.7",
"dotenv-expand": "^11.0.7",
"maildev": "github:timshel/maildev#3.0.2"
},
"dependencies": {
"mysql2": "^3.12.0",
"otpauth": "^9.3.6",
"pg": "^8.13.1"
}
}

137
playwright/playwright.config.ts
View File

@ -0,0 +1,137 @@
import { defineConfig, devices } from '@playwright/test';
import { exec } from 'node:child_process';
const utils = require('./global-utils');
utils.loadEnv();
/**
* See https://playwright.dev/docs/test-configuration.
*/
export default defineConfig({
testDir: './.',
/* Run tests in files in parallel */
fullyParallel: false,
/* Fail the build on CI if you accidentally left test.only in the source code. */
forbidOnly: !!process.env.CI,
/* Retry on CI only */
retries: 0,
workers: 1,
/* Reporter to use. See https://playwright.dev/docs/test-reporters */
reporter: 'html',
timeout: 20 * 1000,
expect: { timeout: 10 * 1000 },
/* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
use: {
/* Base URL to use in actions like `await page.goto('/')`. */
baseURL: process.env.DOMAIN,
browserName: 'firefox',
locale: 'en-GB',
timezoneId: 'Europe/London',
/* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
trace: 'on-first-retry',
viewport: {
width: 1920,
height: 1080
},
video: "on",
},
/* Configure projects for major browsers */
projects: [
{
name: 'mariadb-setup',
testMatch: 'tests/setups/db-setup.ts',
use: { serviceName: "Mariadb" },
teardown: 'mariadb-teardown',
},
{
name: 'mysql-setup',
testMatch: 'tests/setups/db-setup.ts',
use: { serviceName: "Mysql" },
teardown: 'mysql-teardown',
},
{
name: 'postgres-setup',
testMatch: 'tests/setups/db-setup.ts',
use: { serviceName: "Postgres" },
teardown: 'postgres-teardown',
},
{
name: 'sso-setup',
testMatch: 'tests/setups/sso-setup.ts',
teardown: 'sso-teardown',
},
{
name: 'mariadb',
testMatch: 'tests/*.spec.ts',
testIgnore: 'tests/sso_*.spec.ts',
dependencies: ['mariadb-setup'],
},
{
name: 'mysql',
testMatch: 'tests/*.spec.ts',
testIgnore: 'tests/sso_*.spec.ts',
dependencies: ['mysql-setup'],
},
{
name: 'postgres',
testMatch: 'tests/*.spec.ts',
testIgnore: 'tests/sso_*.spec.ts',
dependencies: ['postgres-setup'],
},
{
name: 'sqlite',
testMatch: 'tests/*.spec.ts',
testIgnore: 'tests/sso_*.spec.ts',
},
{
name: 'sso-mariadb',
testMatch: 'tests/sso_*.spec.ts',
dependencies: ['sso-setup', 'mariadb-setup'],
},
{
name: 'sso-mysql',
testMatch: 'tests/sso_*.spec.ts',
dependencies: ['sso-setup', 'mysql-setup'],
},
{
name: 'sso-postgres',
testMatch: 'tests/sso_*.spec.ts',
dependencies: ['sso-setup', 'postgres-setup'],
},
{
name: 'sso-sqlite',
testMatch: 'tests/sso_*.spec.ts',
dependencies: ['sso-setup'],
},
{
name: 'mariadb-teardown',
testMatch: 'tests/setups/db-teardown.ts',
use: { serviceName: "Mariadb" },
},
{
name: 'mysql-teardown',
testMatch: 'tests/setups/db-teardown.ts',
use: { serviceName: "Mysql" },
},
{
name: 'postgres-teardown',
testMatch: 'tests/setups/db-teardown.ts',
use: { serviceName: "Postgres" },
},
{
name: 'sso-teardown',
testMatch: 'tests/setups/sso-teardown.ts',
},
],
globalSetup: require.resolve('./global-setup'),
});

90
playwright/test.env
View File

@ -0,0 +1,90 @@
##################################################################
### Shared Playwright conf test file Vaultwarden and Databases ###
##################################################################
ENV=test
DC_ENV_FILE=test.env
COMPOSE_IGNORE_ORPHANS=True
DOCKER_BUILDKIT=1
#####################
# Playwright Config #
#####################
PW_KEEP_SERVICE_RUNNNING=${PW_KEEP_SERVICE_RUNNNING:-false}
VAULTWARDEN_SMTP_FROM=vaultwarden@playwright.test
#####################
# Maildev Config #
#####################
MAILDEV_HTTP_PORT=1081
MAILDEV_SMTP_PORT=1026
MAILDEV_HOST=127.0.0.1
################
# Users Config #
################
TEST_USER=test
TEST_USER_PASSWORD=Master Password
TEST_USER_MAIL=${TEST_USER}@example.com
TEST_USER2=test2
TEST_USER2_PASSWORD=Master Password
TEST_USER2_MAIL=${TEST_USER2}@example.com
TEST_USER3=test3
TEST_USER3_PASSWORD=Master Password
TEST_USER3_MAIL=${TEST_USER3}@example.com
###################
# Keycloak Config #
###################
KEYCLOAK_ADMIN=admin
KEYCLOAK_ADMIN_PASSWORD=${KEYCLOAK_ADMIN}
KC_HTTP_HOST=127.0.0.1
KC_HTTP_PORT=8081
# Script parameters (use Keycloak and VaultWarden config too)
TEST_REALM=test
DUMMY_REALM=dummy
DUMMY_AUTHORITY=http://${KC_HTTP_HOST}:${KC_HTTP_PORT}/realms/${DUMMY_REALM}
######################
# Vaultwarden Config #
######################
ROCKET_PORT=8003
DOMAIN=http://127.0.0.1:${ROCKET_PORT}
SMTP_SECURITY=off
SMTP_PORT=${MAILDEV_SMTP_PORT}
SMTP_FROM_NAME=Vaultwarden
SMTP_TIMEOUT=5
SSO_CLIENT_ID=VaultWarden
SSO_CLIENT_SECRET=VaultWarden
SSO_AUTHORITY=http://${KC_HTTP_HOST}:${KC_HTTP_PORT}/realms/${TEST_REALM}
SSO_PKCE=true
###########################
# Docker MariaDb container#
###########################
MARIADB_PORT=3307
MARIADB_ROOT_PASSWORD=vaultwarden
MARIADB_USER=vaultwarden
MARIADB_PASSWORD=vaultwarden
MARIADB_DATABASE=vaultwarden
###########################
# Docker Mysql container#
###########################
MYSQL_PORT=3309
MYSQL_ROOT_PASSWORD=vaultwarden
MYSQL_USER=vaultwarden
MYSQL_PASSWORD=vaultwarden
MYSQL_DATABASE=vaultwarden
############################
# Docker Postgres container#
############################
POSTGRES_PORT=5433
POSTGRES_USER=vaultwarden
POSTGRES_PASSWORD=vaultwarden
POSTGRES_DB=vaultwarden

163
playwright/tests/login.smtp.spec.ts
View File

@ -0,0 +1,163 @@
import { test, expect, type TestInfo } from '@playwright/test';
import { MailDev } from 'maildev';
const utils = require('../global-utils');
import { createAccount, logUser } from './setups/user';
let users = utils.loadEnv();
let mailserver;
test.beforeAll('Setup', async ({ browser }, testInfo: TestInfo) => {
mailserver = new MailDev({
port: process.env.MAILDEV_SMTP_PORT,
web: { port: process.env.MAILDEV_HTTP_PORT },
})
await mailserver.listen();
await utils.startVaultwarden(browser, testInfo, {
SMTP_HOST: process.env.MAILDEV_HOST,
SMTP_FROM: process.env.VAULTWARDEN_SMTP_FROM,
});
});
test.afterAll('Teardown', async ({}) => {
utils.stopVaultwarden();
if( mailserver ){
await mailserver.close();
}
});
test('Account creation', async ({ page }) => {
const emails = mailserver.iterator(users.user1.email);
await createAccount(test, page, users.user1);
const { value: created } = await emails.next();
expect(created.subject).toBe("Welcome");
expect(created.from[0]?.address).toBe(process.env.VAULTWARDEN_SMTP_FROM);
// Back to the login page
await expect(page).toHaveTitle('Vaultwarden Web');
await expect(page.getByTestId("toast-message")).toHaveText(/Your new account has been created/);
await page.getByRole('button', { name: 'Continue' }).click();
// Unlock page
await page.getByLabel('Master password').fill(users.user1.password);
await page.getByRole('button', { name: 'Log in with master password' }).click();
// We are now in the default vault page
await expect(page).toHaveTitle(/Vaultwarden Web/);
const { value: logged } = await emails.next();
expect(logged.subject).toBe("New Device Logged In From Firefox");
expect(logged.to[0]?.address).toBe(process.env.TEST_USER_MAIL);
expect(logged.from[0]?.address).toBe(process.env.VAULTWARDEN_SMTP_FROM);
emails.return();
});
test('Login', async ({ context, page }) => {
const emails = mailserver.iterator(users.user1.email);
await logUser(test, page, users.user1);
await test.step('new device email', async () => {
const { value: logged } = await emails.next();
expect(logged.subject).toBe("New Device Logged In From Firefox");
expect(logged.from[0]?.address).toBe(process.env.VAULTWARDEN_SMTP_FROM);
});
await test.step('verify email', async () => {
await page.getByText('Verify your account\'s email').click();
await expect(page.getByText('Verify your account\'s email')).toBeVisible();
await page.getByRole('button', { name: 'Send email' }).click();
// Close the toast message
await expect(page.getByTestId("toast-message")).toHaveText(/Check your email inbox/);
await page.locator('#toast-container').getByRole('button').click();
await expect(page.getByTestId("toast-message")).toHaveCount(0);
const { value: verify } = await emails.next();
expect(verify.subject).toBe("Verify Your Email");
expect(verify.from[0]?.address).toBe(process.env.VAULTWARDEN_SMTP_FROM);
const page2 = await context.newPage();
await page2.setContent(verify.html);
const link = await page2.getByTestId("verify").getAttribute("href");
await page2.close();
await page.goto(link);
await expect(page.getByTestId("toast-message")).toHaveText("Account email verified");
});
emails.return();
});
test('Activaite 2fa', async ({ context, page }) => {
const emails = mailserver.buffer(users.user1.email);
await logUser(test, page, users.user1);
await page.getByRole('button', { name: users.user1.name }).click();
await page.getByRole('menuitem', { name: 'Account settings' }).click();
await page.getByRole('link', { name: 'Security' }).click();
await page.getByRole('link', { name: 'Two-step login' }).click();
await page.locator('li').filter({ hasText: 'Email' }).getByRole('button').click();
await page.getByLabel('Master password (required)').fill(users.user1.password);
await page.getByRole('button', { name: 'Continue' }).click();
await page.getByRole('button', { name: 'Send email' }).click();
const codeMail = await emails.next((mail) => mail.subject === "Vaultwarden Login Verification Code");
const page2 = await context.newPage();
await page2.setContent(codeMail.html);
const code = await page2.getByTestId("2fa").innerText();
await page2.close();
await page.getByLabel('2. Enter the resulting 6').fill(code);
await page.getByRole('button', { name: 'Turn on' }).click();
await page.getByRole('heading', { name: 'Turned on', exact: true });
emails.close();
});
test('2fa', async ({ context, page }) => {
const emails = mailserver.buffer(users.user1.email);
await test.step('login', async () => {
await page.goto('/');
await page.getByLabel(/Email address/).fill(users.user1.email);
await page.getByRole('button', { name: 'Continue' }).click();
await page.getByLabel('Master password').fill(users.user1.password);
await page.getByRole('button', { name: 'Log in with master password' }).click();
const codeMail = await emails.next((mail) => mail.subject === "Vaultwarden Login Verification Code");
const page2 = await context.newPage();
await page2.setContent(codeMail.html);
const code = await page2.getByTestId("2fa").innerText();
await page2.close();
await page.getByLabel('Verification code').fill(code);
await page.getByRole('button', { name: 'Continue' }).click();
await expect(page).toHaveTitle(/Vaultwarden Web/);
})
await test.step('disable', async () => {
await page.getByRole('button', { name: 'Test' }).click();
await page.getByRole('menuitem', { name: 'Account settings' }).click();
await page.getByRole('link', { name: 'Security' }).click();
await page.getByRole('link', { name: 'Two-step login' }).click();
await page.locator('li').filter({ hasText: 'Email' }).getByRole('button').click();
await page.getByLabel('Master password (required)').click();
await page.getByLabel('Master password (required)').fill(users.user1.password);
await page.getByRole('button', { name: 'Continue' }).click();
await page.getByRole('button', { name: 'Turn off' }).click();
await page.getByRole('button', { name: 'Yes' }).click();
await expect(page.getByTestId("toast-message")).toHaveText(/Two-step login provider turned off/);
});
emails.close();
});

94
playwright/tests/login.spec.ts
View File

@ -0,0 +1,94 @@
import { test, expect, type Page, type TestInfo } from '@playwright/test';
import * as OTPAuth from "otpauth";
import * as utils from "../global-utils";
import { createAccount, logUser } from './setups/user';
let users = utils.loadEnv();
let totp;
test.beforeAll('Setup', async ({ browser }, testInfo: TestInfo) => {
await utils.startVaultwarden(browser, testInfo, {});
});
test.afterAll('Teardown', async ({}, testInfo: TestInfo) => {
utils.stopVaultwarden(testInfo);
});
test('Account creation', async ({ page }) => {
// Landing page
await createAccount(test, page, users.user1);
await page.getByRole('button', { name: 'Continue' }).click();
// Unlock page
await page.getByLabel('Master password').fill(users.user1.password);
await page.getByRole('button', { name: 'Log in with master password' }).click();
// We are now in the default vault page
await expect(page).toHaveTitle(/Vaultwarden Web/);
});
test('Master password login', async ({ page }) => {
await logUser(test, page, users.user1);
});
test('Authenticator 2fa', async ({ context, page }) => {
let totp;
await test.step('Login', async () => {
await logUser(test, page, users.user1);
});
await test.step('Activate', async () => {
await page.getByRole('button', { name: users.user1.name }).click();
await page.getByRole('menuitem', { name: 'Account settings' }).click();
await page.getByRole('link', { name: 'Security' }).click();
await page.getByRole('link', { name: 'Two-step login' }).click();
await page.locator('li').filter({ hasText: 'TOTP Authenticator' }).getByRole('button').click();
await page.getByLabel('Master password (required)').fill(users.user1.password);
await page.getByRole('button', { name: 'Continue' }).click();
const secret = await page.getByLabel('Key').innerText();
totp = new OTPAuth.TOTP({ secret, period: 30 });
await page.getByLabel('Verification code (required)').fill(totp.generate());
await page.getByRole('button', { name: 'Turn on' }).click();
await page.getByRole('heading', { name: 'Turned on', exact: true });
await page.getByLabel('Close').click();
})
await test.step('logout', async () => {
await page.getByRole('button', { name: users.user1.name }).click();
await page.getByRole('menuitem', { name: 'Log out' }).click();
});
await test.step('login', async () => {
let timestamp = Date.now(); // Need to use the next token
timestamp = timestamp + (totp.period - (Math.floor(timestamp / 1000) % totp.period) + 1) * 1000;
await page.getByLabel(/Email address/).fill(users.user1.email);
await page.getByRole('button', { name: 'Continue' }).click();
await page.getByLabel('Master password').fill(users.user1.password);
await page.getByRole('button', { name: 'Log in with master password' }).click();
await page.getByLabel('Verification code').fill(totp.generate({timestamp}));
await page.getByRole('button', { name: 'Continue' }).click();
await expect(page).toHaveTitle(/Vaultwarden Web/);
});
await test.step('disable', async () => {
await page.getByRole('button', { name: 'Test' }).click();
await page.getByRole('menuitem', { name: 'Account settings' }).click();
await page.getByRole('link', { name: 'Security' }).click();
await page.getByRole('link', { name: 'Two-step login' }).click();
await page.locator('li').filter({ hasText: 'TOTP Authenticator' }).getByRole('button').click();
await page.getByLabel('Master password (required)').click();
await page.getByLabel('Master password (required)').fill(users.user1.password);
await page.getByRole('button', { name: 'Continue' }).click();
await page.getByRole('button', { name: 'Turn off' }).click();
await page.getByRole('button', { name: 'Yes' }).click();
await expect(page.getByTestId("toast-message")).toHaveText(/Two-step login provider turned off/);
});
});

164
playwright/tests/organization.spec.ts
View File

@ -0,0 +1,164 @@
import { test, expect, type TestInfo } from '@playwright/test';
import { MailDev } from 'maildev';
import * as utils from "../global-utils";
import { createAccount, logUser } from './setups/user';
let users = utils.loadEnv();
let mailserver, user1Mails, user2Mails, user3Mails;
test.beforeAll('Setup', async ({ browser }, testInfo: TestInfo) => {
mailserver = new MailDev({
port: process.env.MAILDEV_SMTP_PORT,
web: { port: process.env.MAILDEV_HTTP_PORT },
})
await mailserver.listen();
await utils.startVaultwarden(browser, testInfo, {
SMTP_HOST: process.env.MAILDEV_HOST,
SMTP_FROM: process.env.VAULTWARDEN_SMTP_FROM,
});
user1Mails = mailserver.iterator(users.user1.email);
user2Mails = mailserver.iterator(users.user2.email);
user3Mails = mailserver.iterator(users.user3.email);
});
test.afterAll('Teardown', async ({}, testInfo: TestInfo) => {
utils.stopVaultwarden(testInfo);
utils.closeMails(mailserver, [user1Mails, user2Mails, user3Mails]);
});
test('Create user3', async ({ page }) => {
await createAccount(test, page, users.user3, user3Mails);
});
test('Invite users', async ({ page }) => {
await createAccount(test, page, users.user1, user1Mails);
await logUser(test, page, users.user1, user1Mails);
await test.step('Create Org', async () => {
await page.getByRole('link', { name: 'New organisation' }).click();
await page.getByLabel('Organisation name (required)').fill('Test');
await page.getByRole('button', { name: 'Submit' }).click();
await page.locator('div').filter({ hasText: 'Members' }).nth(2).click();
});
await test.step('Invite user2', async () => {
await page.getByRole('button', { name: 'Invite member' }).click();
await page.getByLabel('Email (required)').fill(users.user2.email);
await page.getByRole('tab', { name: 'Collections' }).click();
await page.getByLabel('Permission').selectOption('edit');
await page.getByLabel('Select collections').click();
await page.getByLabel('Options list').getByText('Default collection').click();
await page.getByRole('button', { name: 'Save' }).click();
await expect(page.getByTestId("toast-message")).toHaveText('User(s) invited');
});
await test.step('Invite user3', async () => {
await page.getByRole('button', { name: 'Invite member' }).click();
await page.getByLabel('Email (required)').fill(users.user3.email);
await page.getByRole('tab', { name: 'Collections' }).click();
await page.getByLabel('Permission').selectOption('edit');
await page.getByLabel('Select collections').click();
await page.getByLabel('Options list').getByText('Default collection').click();
await page.getByRole('button', { name: 'Save' }).click();
await expect(page.getByTestId("toast-message")).toHaveText('User(s) invited');
});
});
test('invited with new account', async ({ page }) => {
const { value: invited } = await user2Mails.next();
expect(invited.subject).toContain("Join Test")
await test.step('Create account', async () => {
await page.setContent(invited.html);
const link = await page.getByTestId("invite").getAttribute("href");
await page.goto(link);
await expect(page).toHaveTitle(/Create account | Vaultwarden Web/);
await page.getByLabel('Name').fill(users.user2.name);
await page.getByLabel('Master password\n (required)', { exact: true }).fill(users.user2.password);
await page.getByLabel('Re-type master password').fill(users.user2.password);
await page.getByRole('button', { name: 'Create account' }).click();
// Back to the login page
await expect(page).toHaveTitle('Vaultwarden Web');
await expect(page.getByTestId("toast-message")).toHaveText(/Your new account has been created/);
const { value: welcome } = await user2Mails.next();
expect(welcome.subject).toContain("Welcome")
});
await test.step('Login', async () => {
await page.getByLabel(/Email address/).fill(users.user2.email);
await page.getByRole('button', { name: 'Continue' }).click();
// Unlock page
await page.getByLabel('Master password').fill(users.user2.password);
await page.getByRole('button', { name: 'Log in with master password' }).click();
// We are now in the default vault page
await expect(page).toHaveTitle(/Vaultwarden Web/);
await expect(page.getByTestId("toast-title")).toHaveText("Invitation accepted");
const { value: logged } = await user2Mails.next();
expect(logged.subject).toContain("New Device Logged");
});
const { value: accepted } = await user1Mails.next();
expect(accepted.subject).toContain("Invitation to Test accepted")
});
test('invited with existing account', async ({ page }) => {
const { value: invited } = await user3Mails.next();
expect(invited.subject).toContain("Join Test")
await page.setContent(invited.html);
const link = await page.getByTestId("invite").getAttribute("href");
await page.goto(link);
// We should be on login page with email prefilled
await expect(page).toHaveTitle(/Vaultwarden Web/);
await page.getByRole('button', { name: 'Continue' }).click();
// Unlock page
await page.getByLabel('Master password').fill(users.user3.password);
await page.getByRole('button', { name: 'Log in with master password' }).click();
// We are now in the default vault page
await expect(page).toHaveTitle(/Vaultwarden Web/);
await expect(page.getByTestId("toast-title")).toHaveText("Invitation accepted");
const { value: logged } = await user3Mails.next();
expect(logged.subject).toContain("New Device Logged")
const { value: accepted } = await user1Mails.next();
expect(accepted.subject).toContain("Invitation to Test accepted")
});
test('Confirm invited user', async ({ page }) => {
await logUser(test, page, users.user1, user1Mails);
await page.getByLabel('Switch products').click();
await page.getByRole('link', { name: ' Admin Console' }).click();
await page.getByRole('link', { name: 'Members' }).click();
await test.step('Accept user2', async () => {
await page.getByRole('row', { name: users.user2.name }).getByLabel('Options').click();
await page.getByRole('menuitem', { name: 'Confirm' }).click();
await page.getByRole('button', { name: 'Confirm' }).click();
await expect(page.getByTestId("toast-message")).toHaveText(/confirmed/);
const { value: logged } = await user2Mails.next();
expect(logged.subject).toContain("Invitation to Test confirmed");
});
});
test('Organization is visible', async ({ page }) => {
await logUser(test, page, users.user2, user2Mails);
await page.getByLabel('vault: Test').click();
await expect(page.getByLabel('Filter: Default collection')).toBeVisible();
});

7
playwright/tests/setups/db-setup.ts
View File

@ -0,0 +1,7 @@
import { test } from './db-test';
const utils = require('../../global-utils');
test('DB start', async ({ serviceName }) => {
utils.startComposeService(serviceName);
});

11
playwright/tests/setups/db-teardown.ts
View File

@ -0,0 +1,11 @@
import { test } from './db-test';
const utils = require('../../global-utils');
utils.loadEnv();
test('DB teardown ?', async ({ serviceName }) => {
if( process.env.PW_KEEP_SERVICE_RUNNNING !== "true" ) {
utils.stopComposeService(serviceName);
}
});

9
playwright/tests/setups/db-test.ts
View File

@ -0,0 +1,9 @@
import { test as base } from '@playwright/test';
export type TestOptions = {
serviceName: string;
};
export const test = base.extend<TestOptions>({
serviceName: ['', { option: true }],
});

19
playwright/tests/setups/sso-setup.ts
View File

@ -0,0 +1,19 @@
import { test, expect, type TestInfo } from '@playwright/test';
const { exec } = require('node:child_process');
const utils = require('../../global-utils');
utils.loadEnv();
test.beforeAll('Setup', async () => {
console.log("Starting Keycloak");
exec(`docker compose --profile keycloak --env-file test.env up`);
});
test('Keycloak is up', async ({ page }) => {
test.setTimeout(60000);
await utils.waitFor(process.env.SSO_AUTHORITY, page.context().browser());
// Dummy authority is created at the end of the setup
await utils.waitFor(process.env.DUMMY_AUTHORITY, page.context().browser());
console.log(`Keycloak running on: ${process.env.SSO_AUTHORITY}`);
});

15
playwright/tests/setups/sso-teardown.ts
View File

@ -0,0 +1,15 @@
import { test, type FullConfig } from '@playwright/test';
const { execSync } = require('node:child_process');
const utils = require('../../global-utils');
utils.loadEnv();
test('Keycloak teardown', async () => {
if( process.env.PW_KEEP_SERVICE_RUNNNING === "true" ) {
console.log("Keep Keycloak running");
} else {
console.log("Keycloak stopping");
execSync(`docker compose --profile keycloak --env-file test.env stop Keycloak`);
}
});

111
playwright/tests/setups/sso.ts
View File

@ -0,0 +1,111 @@
import { expect, type Page, Test } from '@playwright/test';
import { type MailBuffer, MailServer } from 'maildev';
/**
* If a MailBuffer is passed it will be used and consume the expected emails
*/
export async function logNewUser(
test: Test,
page: Page,
user: { email: string, name: string, password: string },
options: { mailBuffer?: MailBuffer, mailServer?: MailServer } = {}
) {
let mailBuffer = options.mailBuffer ?? options.mailServer?.buffer(user.email);
try {
await test.step('Create user', async () => {
await test.step('Landing page', async () => {
await page.goto('/');
await page.getByLabel(/Email address/).fill(user.email);
await page.getByRole('button', 'Continue').click();
});
await test.step('SSo start page', async () => {
await page.getByRole('link', { name: /Enterprise single sign-on/ }).click();
});
await test.step('Keycloak login', async () => {
await expect(page.getByRole('heading', { name: 'Sign in to your account' })).toBeVisible();
await page.getByLabel(/Username/).fill(user.name);
await page.getByLabel('Password', { exact: true }).fill(user.password);
await page.getByRole('button', { name: 'Sign In' }).click();
});
await test.step('Create Vault account', async () => {
await expect(page.getByText('Set master password')).toBeVisible();
await page.getByLabel('Master password', { exact: true }).fill(user.password);
await page.getByLabel('Re-type master password').fill(user.password);
await page.getByRole('button', { name: 'Submit' }).click();
});
await test.step('Default vault page', async () => {
await expect(page).toHaveTitle(/Vaultwarden Web/);
await expect(page.getByTitle('All vaults', { exact: true })).toBeVisible();
});
if( mailBuffer ){
await test.step('Check emails', async () => {
await expect(mailBuffer.next((m) => m.subject.includes("New Device Logged"))).resolves.toBeDefined();
await expect(mailBuffer.next((m) => m.subject === "Master Password Has Been Changed")).resolves.toBeDefined();
});
}
});
} finally {
if( options.mailServer ){
mailBuffer.close();
}
}
}
/**
* If a MailBuffer is passed it will be used and consume the expected emails
*/
export async function logUser(
test: Test,
page: Page,
user: { email: string, password: string },
options: { mailBuffer ?: MailBuffer, mailServer?: MailServer} = {}
) {
let mailBuffer = options.mailBuffer ?? options.mailServer?.buffer(user.email);
try {
await test.step('Log user', async () => {
await test.step('Landing page', async () => {
await page.goto('/');
await page.getByLabel(/Email address/).fill(user.email);
await page.getByRole('button', 'Continue').click();
});
await test.step('SSo start page', async () => {
await page.getByRole('link', { name: /Enterprise single sign-on/ }).click();
});
await test.step('Keycloak login', async () => {
await expect(page.getByRole('heading', { name: 'Sign in to your account' })).toBeVisible();
await page.getByLabel(/Username/).fill(user.name);
await page.getByLabel('Password', { exact: true }).fill(user.password);
await page.getByRole('button', { name: 'Sign In' }).click();
});
await test.step('Unlock vault', async () => {
await expect(page).toHaveTitle('Vaultwarden Web');
await expect(page.getByRole('heading', { name: 'Your vault is locked' })).toBeVisible();
await page.getByLabel('Master password').fill(user.password);
await page.getByRole('button', { name: 'Unlock' }).click();
});
await test.step('Default vault page', async () => {
await expect(page).toHaveTitle(/Vaultwarden Web/);
await expect(page.getByTitle('All vaults', { exact: true })).toBeVisible();
});
if( options.emails ){
await test.step('Check email', async () => {
await expect(mailBuffer.next((m) => m.subject.includes("New Device Logged"))).resolves.toBeDefined();
});
}
});
} finally {
if( options.mailServer ){
mailBuffer.close();
}
}
}

47
playwright/tests/setups/user.ts
View File

@ -0,0 +1,47 @@
import { expect, type Browser,Page } from '@playwright/test';
export async function createAccount(test, page: Page, user: { email: string, name: string, password: string }, emails) {
await test.step('Create user', async () => {
// Landing page
await page.goto('/');
await page.getByRole('link', { name: 'Create account' }).click();
// Back to Vault create account
await expect(page).toHaveTitle(/Create account | Vaultwarden Web/);
await page.getByLabel(/Email address/).fill(user.email);
await page.getByLabel('Name').fill(user.name);
await page.getByLabel('Master password\n (required)', { exact: true }).fill(user.password);
await page.getByLabel('Re-type master password').fill(user.password);
await page.getByRole('button', { name: 'Create account' }).click();
// Back to the login page
await expect(page).toHaveTitle('Vaultwarden Web');
await expect(page.getByTestId("toast-message")).toHaveText(/Your new account has been created/);
if( emails ){
const { value: welcome } = await emails.next();
expect(welcome.subject).toContain("Welcome");
}
});
}
export async function logUser(test, page: Page, user: { email: string, password: string }, emails) {
await test.step('Log user', async () => {
// Landing page
await page.goto('/');
await page.getByLabel(/Email address/).fill(user.email);
await page.getByRole('button', { name: 'Continue' }).click();
// Unlock page
await page.getByLabel('Master password').fill(user.password);
await page.getByRole('button', { name: 'Log in with master password' }).click();
// We are now in the default vault page
await expect(page).toHaveTitle(/Vaultwarden Web/);
if( emails ){
const { value: logged } = await emails.next();
expect(logged.subject).toContain("New Device Logged");
}
});
}

78
playwright/tests/sso_login.spec.ts
View File

@ -0,0 +1,78 @@
import { test, expect, type TestInfo } from '@playwright/test';
import { logNewUser, logUser } from './setups/sso';
import * as utils from "../global-utils";
let users = utils.loadEnv();
test.beforeAll('Setup', async ({ browser }, testInfo: TestInfo) => {
await utils.startVaultwarden(browser, testInfo, {
SSO_ENABLED: true,
SSO_ONLY: false
});
});
test.afterAll('Teardown', async ({}) => {
utils.stopVaultwarden();
});
test('Account creation using SSO', async ({ page }) => {
// Landing page
await logNewUser(test, page, users.user1);
});
test('SSO login', async ({ page }) => {
await logUser(test, page, users.user1);
});
test('Non SSO login', async ({ page }) => {
// Landing page
await page.goto('/');
await page.getByLabel(/Email address/).fill(users.user1.email);
await page.getByRole('button', { name: 'Continue' }).click();
// Unlock page
await page.getByLabel('Master password').fill(users.user1.password);
await page.getByRole('button', { name: 'Log in with master password' }).click();
// We are now in the default vault page
await expect(page).toHaveTitle(/Vaultwarden Web/);
});
test('Non SSO login Failure', async ({ page, browser }, testInfo: TestInfo) => {
await utils.restartVaultwarden(page, testInfo, {
SSO_ENABLED: true,
SSO_ONLY: true
}, false);
// Landing page
await page.goto('/');
await page.getByLabel(/Email address/).fill(users.user1.email);
await page.getByRole('button', { name: 'Continue' }).click();
// Unlock page
await page.getByLabel('Master password').fill(users.user1.password);
await page.getByRole('button', { name: 'Log in with master password' }).click();
// An error should appear
await page.getByLabel('SSO sign-in is required')
// Check the selector for the next test
await expect(page.getByRole('link', { name: /Enterprise single sign-on/ })).toHaveCount(1);
});
test('No SSO login', async ({ page }, testInfo: TestInfo) => {
await utils.restartVaultwarden(page, testInfo, {
SSO_ENABLED: false
}, false);
// Landing page
await page.goto('/');
await page.getByLabel(/Email address/).fill(users.user1.email);
await page.getByRole('button', { name: 'Continue' }).click();
// No SSO button (rely on a correct selector checked in previous test)
await page.getByLabel('Master password');
await expect(page.getByRole('link', { name: /Enterprise single sign-on/ })).toHaveCount(0);
});

142
playwright/tests/sso_organization.spec.ts
View File

@ -0,0 +1,142 @@
import { test, expect, type TestInfo } from '@playwright/test';
import { MailDev } from 'maildev';
import * as utils from "../global-utils";
import { logNewUser, logUser } from './setups/sso';
let users = utils.loadEnv();
let mailServer, mail1Buffer, mail2Buffer, mail3Buffer;
test.beforeAll('Setup', async ({ browser }, testInfo: TestInfo) => {
mailServer = new MailDev({
port: process.env.MAILDEV_SMTP_PORT,
web: { port: process.env.MAILDEV_HTTP_PORT },
})
await mailServer.listen();
await utils.startVaultwarden(browser, testInfo, {
SMTP_HOST: process.env.MAILDEV_HOST,
SMTP_FROM: process.env.VAULTWARDEN_SMTP_FROM,
SSO_ENABLED: true,
SSO_ONLY: true,
});
mail1Buffer = mailServer.buffer(users.user1.email);
mail2Buffer = mailServer.buffer(users.user2.email);
mail3Buffer = mailServer.buffer(users.user3.email);
});
test.afterAll('Teardown', async ({}) => {
utils.stopVaultwarden();
[mailServer, mail1Buffer, mail2Buffer, mail3Buffer].map((m) => m?.close());
});
test('Create user2', async ({ page }) => {
await logNewUser(test, page, users.user2, { mailBuffer: mail2Buffer });
});
test('Invite users', async ({ page }) => {
await logNewUser(test, page, users.user1, { mailBuffer: mail1Buffer });
await test.step('Create Org', async () => {
await page.getByRole('link', { name: 'New organisation' }).click();
await page.getByLabel('Organisation name (required)').fill('Test');
await page.getByRole('button', { name: 'Submit' }).click();
await page.locator('div').filter({ hasText: 'Members' }).nth(2).click();
});
await test.step('Invite user2', async () => {
await page.getByRole('button', { name: 'Invite member' }).click();
await page.getByLabel('Email (required)').fill(users.user2.email);
await page.getByRole('tab', { name: 'Collections' }).click();
await page.getByLabel('Permission').selectOption('edit');
await page.getByLabel('Select collections').click();
await page.getByLabel('Options list').getByText('Default collection').click();
await page.getByRole('button', { name: 'Save' }).click();
await expect(page.getByTestId("toast-message")).toHaveText('User(s) invited');
});
await test.step('Invite user3', async () => {
await page.getByRole('button', { name: 'Invite member' }).click();
await page.getByLabel('Email (required)').fill(users.user3.email);
await page.getByRole('tab', { name: 'Collections' }).click();
await page.getByLabel('Permission').selectOption('edit');
await page.getByLabel('Select collections').click();
await page.getByLabel('Options list').getByText('Default collection').click();
await page.getByRole('button', { name: 'Save' }).click();
await expect(page.getByTestId("toast-message")).toHaveText('User(s) invited');
});
});
test('invited with existing account', async ({ page }) => {
const link = await test.step('Extract email link', async () => {
const invited = await mail2Buffer.next((m) => m.subject === "Join Test");
await page.setContent(invited.html);
return await page.getByTestId("invite").getAttribute("href");
});
await test.step('Redirect to Keycloak', async () => {
await page.goto(link);
});
await test.step('Keycloak login', async () => {
await expect(page.getByRole('heading', { name: 'Sign in to your account' })).toBeVisible();
await page.getByLabel(/Username/).fill(users.user2.name);
await page.getByLabel('Password', { exact: true }).fill(users.user2.password);
await page.getByRole('button', { name: 'Sign In' }).click();
});
await test.step('Unlock vault', async () => {
await expect(page).toHaveTitle('Vaultwarden Web');
await page.getByLabel('Master password').fill(users.user2.password);
await page.getByRole('button', { name: 'Unlock' }).click();
});
await test.step('Default vault page', async () => {
await expect(page).toHaveTitle(/Vaultwarden Web/);
await expect(page.getByTestId("toast-title")).toHaveText("Invitation accepted");
});
await test.step('Check mails', async () => {
await expect(mail2Buffer.next((m) => m.subject.includes("New Device Logged"))).resolves.toBeDefined();
await expect(mail1Buffer.next((m) => m.subject === "Invitation to Test accepted")).resolves.toBeDefined();
});
});
test('invited with new account', async ({ page }) => {
const link = await test.step('Extract email link', async () => {
const invited = await mail3Buffer.next((m) => m.subject === "Join Test");
await page.setContent(invited.html);
return await page.getByTestId("invite").getAttribute("href");
});
await test.step('Redirect to Keycloak', async () => {
await page.goto(link);
});
await test.step('Keycloak login', async () => {
await expect(page.getByRole('heading', { name: 'Sign in to your account' })).toBeVisible();
await page.getByLabel(/Username/).fill(users.user3.name);
await page.getByLabel('Password', { exact: true }).fill(users.user3.password);
await page.getByRole('button', { name: 'Sign In' }).click();
});
await test.step('Create Vault account', async () => {
await expect(page.getByText('Set master password')).toBeVisible();
await page.getByLabel('Master password', { exact: true }).fill(users.user3.password);
await page.getByLabel('Re-type master password').fill(users.user3.password);
await page.getByRole('button', { name: 'Submit' }).click();
});
await test.step('Default vault page', async () => {
await expect(page).toHaveTitle(/Vaultwarden Web/);
await expect(page.getByTestId("toast-title")).toHaveText("Invitation accepted");
});
await test.step('Check mails', async () => {
await expect(mail3Buffer.next((m) => m.subject.includes("New Device Logged"))).resolves.toBeDefined();
await expect(mail1Buffer.next((m) => m.subject === "Invitation to Test accepted")).resolves.toBeDefined();
});
});

2
src/static/templates/email/send_org_invite.html.hbs
View File

@ -9,7 +9,7 @@ Join {{{org_name}}}
</tr>
<tr style="margin: 0; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; box-sizing: border-box; font-size: 16px; color: #333; line-height: 25px; -webkit-font-smoothing: antialiased; -webkit-text-size-adjust: none;">
<td class="content-block" style="font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; box-sizing: border-box; font-size: 16px; color: #333; line-height: 25px; margin: 0; -webkit-font-smoothing: antialiased; padding: 0 0 10px; -webkit-text-size-adjust: none; text-align: center;" valign="top" align="center">
<a href="{{{url}}}"
<a data-testid="invite" href="{{{url}}}"
clicktracking=off target="_blank" style="color: #ffffff; text-decoration: none; text-align: center; cursor: pointer; display: inline-block; border-radius: 5px; background-color: #3c8dbc; border-color: #3c8dbc; border-style: solid; border-width: 10px 20px; margin: 0; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; box-sizing: border-box; font-size: 16px; line-height: 25px; -webkit-font-smoothing: antialiased; -webkit-text-size-adjust: none;">
Join Organization Now
</a>

2
src/static/templates/email/twofactor_email.html.hbs
View File

@ -4,7 +4,7 @@ Vaultwarden Login Verification Code
<table width="100%" cellpadding="0" cellspacing="0" style="margin: 0; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; box-sizing: border-box; font-size: 16px; color: #333; line-height: 25px; -webkit-font-smoothing: antialiased; -webkit-text-size-adjust: none;">
<tr style="margin: 0; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; box-sizing: border-box; font-size: 16px; color: #333; line-height: 25px; -webkit-font-smoothing: antialiased; -webkit-text-size-adjust: none;">
<td class="content-block" style="font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; box-sizing: border-box; font-size: 16px; color: #333; line-height: 25px; margin: 0; -webkit-font-smoothing: antialiased; padding: 0 0 10px; -webkit-text-size-adjust: none;" valign="top">
Your two-step verification code is: <b>{{token}}</b>
Your two-step verification code is: <b data-testid="2fa">{{token}}</b>
</td>
</tr>
<tr style="margin: 0; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; box-sizing: border-box; font-size: 16px; color: #333; line-height: 25px; -webkit-font-smoothing: antialiased; -webkit-text-size-adjust: none;">

2
src/static/templates/email/verify_email.html.hbs
View File

@ -9,7 +9,7 @@ Verify Your Email
</tr>
<tr style="margin: 0; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; box-sizing: border-box; font-size: 16px; color: #333; line-height: 25px; -webkit-font-smoothing: antialiased; -webkit-text-size-adjust: none;">
<td class="content-block" style="font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; box-sizing: border-box; font-size: 16px; color: #333; line-height: 25px; margin: 0; -webkit-font-smoothing: antialiased; padding: 0 0 10px; -webkit-text-size-adjust: none; text-align: center;" valign="top" align="center">
<a href="{{url}}/#/verify-email/?userId={{user_id}}&token={{token}}"
<a data-testid="verify" href="{{url}}/#/verify-email/?userId={{user_id}}&token={{token}}"
clicktracking=off target="_blank" style="color: #ffffff; text-decoration: none; text-align: center; cursor: pointer; display: inline-block; border-radius: 5px; background-color: #3c8dbc; border-color: #3c8dbc; border-style: solid; border-width: 10px 20px; margin: 0; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; box-sizing: border-box; font-size: 16px; line-height: 25px; -webkit-font-smoothing: antialiased; -webkit-text-size-adjust: none;">
Verify Email Address Now
</a>

Write Preview
Loading…
Cancel
Save