Add playwright tests for all db

1 year ago · 2618730806
22 changed files with 3472 additions and 4 deletions
--- a/playwright/.gitignore
+++ b/playwright/.gitignore
@ -0,0 +1,6 @@
+logs
+node_modules/
+/test-results/
+/playwright-report/
+/playwright/.cache/
+temp
--- a/playwright/README.md
+++ b/playwright/README.md
@ -0,0 +1,88 @@
+# Integration tests
+
+This allows running integration tests using [Playwright](https://playwright.dev/).
+\
+It usse its own [test.env](/test/scenarios/test.env) with different ports to not collide with a running dev instance.
+
+## Install
+
+This rely on `docker` and the `compose` [plugin](https://docs.docker.com/compose/install/).
+Databases (`Mariadb`, `Mysql` and `Postgres`) and `Playwright` will run in containers.
+
+### Running Playwright outside docker
+
+It's possible to run `Playwright` outside of the container, this remove the need to rebuild the image for each change.
+You'll additionally need `nodejs` then run:
+
+```bash
+npm install
+npx playwright install-deps
+npx playwright install firefox
+```
+
+## Usage
+
+To run all the tests:
+
+```bash
+DOCKER_BUILDKIT=1 docker compose --env-file test.env run Playwright
+```
+
+To force a rebuild of the Playwright image:
+```bash
+DOCKER_BUILDKIT=1 docker compose --env-file test.env build Playwright
+```
+
+To access the ui to easily run test individually and debug if needed (will not work in docker):
+
+```bash
+npx playwright test --ui
+```
+
+### DB
+
+Projects are configured to allow to run tests only on specific database.
+\
+You can use:
+
+```bash
+DOCKER_BUILDKIT=1 docker compose --env-file test.env run Playwright test --project=mariadb
+DOCKER_BUILDKIT=1 docker compose --env-file test.env run Playwright test --project=mysql
+DOCKER_BUILDKIT=1 docker compose --env-file test.env run Playwright test --project=postgres
+DOCKER_BUILDKIT=1 docker compose --env-file test.env run Playwright test --project=sqlite
+```
+
+### Running specific tests
+
+To run a whole file you can :
+
+```bash
+DOCKER_BUILDKIT=1 docker compose --env-file test.env run Playwright test --project=sqlite tests/login.spec.ts
+DOCKER_BUILDKIT=1 docker compose --env-file test.env run Playwright test --project=sqlite login
+```
+
+To run only a specifc test (It might fail if it has dependency):
+
+```bash
+DOCKER_BUILDKIT=1 docker compose --env-file test.env run Playwright test --project=sqlite -g "Account creation"
+DOCKER_BUILDKIT=1 docker compose --env-file test.env run Playwright test --project=sqlite tests/login.spec.ts:16
+```
+
+## Writing scenario
+
+When creating new scenario use the recorder to more easily identify elements (in general try to rely on visible hint to identify elements and not hidden ids).
+This does not start the server, you will need to start it manually.
+
+```bash
+npx playwright codegen "http://127.0.0.1:8000"
+```
+
+## Override web-vault
+
+It's possible to change the `web-vault` used by referencing a different `bw_web_builds` commit.
+
+```bash
+export PW_WV_REPO_URL=https://github.com/Timshel/oidc_web_builds.git
+export PW_WV_COMMIT_HASH=8707dc76df3f0cceef2be5bfae37bb29bd17fae6
+DOCKER_BUILDKIT=1 docker compose --env-file test.env build Playwright
+```
--- a/playwright/compose/playwright/Dockerfile
+++ b/playwright/compose/playwright/Dockerfile
@ -0,0 +1,40 @@
+FROM docker.io/library/debian:bookworm-slim
+
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+
+ENV DEBIAN_FRONTEND=noninteractive
+
+RUN apt-get update \
+    && apt-get install -y ca-certificates curl \
+    && curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc \
+    && chmod a+r /etc/apt/keyrings/docker.asc \
+    && echo "deb [signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian bookworm stable" | tee /etc/apt/sources.list.d/docker.list \
+    && apt-get update \
+    && apt-get install -y --no-install-recommends \
+        containerd.io \
+        docker-buildx-plugin \
+        docker-ce \
+        docker-ce-cli \
+        docker-compose-plugin \
+        git \
+        libmariadb-dev-compat \
+        libpq5 \
+        nodejs \
+        npm \
+        openssl \
+    && rm -rf /var/lib/apt/lists/*
+
+RUN mkdir /playwright
+WORKDIR /playwright
+
+COPY package.json .
+RUN npm install && npx playwright install-deps && npx playwright install firefox
+
+COPY docker-compose.yml test.env ./
+COPY compose ./compose
+
+COPY *.ts test.env ./
+COPY tests ./tests
+
+ENTRYPOINT ["/usr/bin/npx", "playwright"]
+CMD ["test"]
--- a/playwright/compose/vaultwarden/Dockerfile
+++ b/playwright/compose/vaultwarden/Dockerfile
@ -0,0 +1,39 @@
+FROM playwright_vaultwarden_prebuilt AS vaultwarden
+
+FROM node:18-bookworm AS build
+
+arg REPO_URL
+arg COMMIT_HASH
+
+ENV REPO_URL=$REPO_URL
+ENV COMMIT_HASH=$COMMIT_HASH
+
+COPY --from=vaultwarden /web-vault /web-vault
+COPY build.sh /build.sh
+RUN /build.sh
+
+######################## RUNTIME IMAGE  ########################
+FROM docker.io/library/debian:bookworm-slim
+
+ENV DEBIAN_FRONTEND=noninteractive
+
+# Create data folder and Install needed libraries
+RUN mkdir /data && \
+    apt-get update && apt-get install -y \
+        --no-install-recommends \
+        ca-certificates \
+        curl \
+        libmariadb-dev-compat \
+        libpq5 \
+        openssl && \
+    rm -rf /var/lib/apt/lists/*
+
+# Copies the files from the context (Rocket.toml file and web-vault)
+# and the binary from the "build" stage to the current stage
+WORKDIR /
+
+COPY --from=vaultwarden /start.sh .
+COPY --from=vaultwarden /vaultwarden .
+COPY --from=build /web-vault ./web-vault
+
+ENTRYPOINT ["/start.sh"]
--- a/playwright/compose/vaultwarden/build.sh
+++ b/playwright/compose/vaultwarden/build.sh
@ -0,0 +1,24 @@
+#!/bin/bash
+
+echo $REPO_URL
+echo $COMMIT_HASH
+
+if [[ ! -z "$REPO_URL" ]] && [[ ! -z "$COMMIT_HASH" ]] ; then
+    rm -rf /web-vault
+
+    mkdir bw_web_builds;
+    cd bw_web_builds;
+
+    git -c init.defaultBranch=main init
+    git remote add origin "$REPO_URL"
+    git fetch --depth 1 origin "$COMMIT_HASH"
+    git -c advice.detachedHead=false checkout FETCH_HEAD
+
+    export VAULT_VERSION=$(cat Dockerfile | grep "ARG VAULT_VERSION" | cut -d "=" -f2)
+    ./scripts/checkout_web_vault.sh
+    ./scripts/patch_web_vault.sh
+    ./scripts/build_web_vault.sh
+    printf '{"version":"%s"}' "$COMMIT_HASH" > ./web-vault/apps/web/build/vw-version.json
+
+    mv ./web-vault/apps/web/build /web-vault
+fi
--- a/playwright/docker-compose.yml
+++ b/playwright/docker-compose.yml
@ -0,0 +1,76 @@
+services:
+  VaultwardenPrebuild:
+    container_name: playwright_vaultwarden_prebuilt
+    image: playwright_vaultwarden_prebuilt
+    build:
+      context: ..
+      dockerfile: Dockerfile
+    entrypoint: /bin/bash
+    restart: "no"
+
+  Vaultwarden:
+    container_name: playwright_vaultwarden-${ENV:-dev}
+    image: playwright_vaultwarden-${ENV:-dev}
+    network_mode: "host"
+    build:
+      context: compose/vaultwarden
+      dockerfile: Dockerfile
+      args:
+        REPO_URL: ${PW_WV_REPO_URL:-}
+        COMMIT_HASH: ${PW_WV_COMMIT_HASH:-}
+    env_file: ${ENV}.env
+    environment:
+      - DATABASE_URL
+      - I_REALLY_WANT_VOLATILE_STORAGE
+      - SMTP_HOST
+      - SMTP_FROM
+      - SMTP_DEBUG
+    restart: "no"
+
+  Playwright:
+    container_name: playwright_playwright
+    image: playwright_playwright
+    network_mode: "host"
+    build:
+      context: .
+      dockerfile: compose/playwright/Dockerfile
+    environment:
+      - PW_WV_REPO_URL
+      - PW_WV_COMMIT_HASH
+    restart: "no"
+    volumes:
+      - /var/run/docker.sock:/var/run/docker.sock
+      - ..:/project
+
+  Mariadb:
+    container_name: playwright_mariadb
+    image: mariadb:11.2.4
+    env_file: test.env
+    healthcheck:
+      test: ["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"]
+      start_period: 10s
+      interval: 10s
+    ports:
+      - ${MARIADB_PORT}:3306
+
+  Mysql:
+    container_name: playwright_mysql
+    image: mysql:8.4.1
+    env_file: test.env
+    healthcheck:
+      test: ["CMD", "mysqladmin" ,"ping", "-h", "localhost"]
+      start_period: 10s
+      interval: 10s
+    ports:
+      - ${MYSQL_PORT}:3306
+
+  Postgres:
+    container_name: playwright_postgres
+    image: postgres:16.3
+    env_file: test.env
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
+      start_period: 20s
+      interval: 30s
+    ports:
+      - ${POSTGRES_PORT}:5432
--- a/playwright/global-setup.ts
+++ b/playwright/global-setup.ts
@ -0,0 +1,22 @@
+import { firefox, type FullConfig } from '@playwright/test';
+import { execSync } from 'node:child_process';
+import fs from 'fs';
+
+const utils = require('./global-utils');
+
+utils.loadEnv();
+
+async function globalSetup(config: FullConfig) {
+    // Are we running in docker and the project is mounted ?
+    const path = (fs.existsSync("/project/playwright/playwright.config.ts") ? "/project/playwright" : ".");
+    execSync(`docker compose --project-directory ${path} --env-file test.env build VaultwardenPrebuild`, {
+        env: { ...process.env },
+        stdio: "inherit"
+    });
+    execSync(`docker compose --project-directory ${path} --env-file test.env build Vaultwarden`, {
+        env: { ...process.env },
+        stdio: "inherit"
+    });
+}
+
+export default globalSetup;
--- a/playwright/global-utils.ts
+++ b/playwright/global-utils.ts
@ -0,0 +1,207 @@
+import { type Browser, type TestInfo } from '@playwright/test';
+import { EventEmitter } from "events";
+import { execSync } from 'node:child_process';
+
+import dotenv from 'dotenv';
+import dotenvExpand from 'dotenv-expand';
+
+const fs = require("fs");
+const { spawn } = require('node:child_process');
+
+function loadEnv(){
+    var myEnv = dotenv.config({ path: 'test.env' });
+    dotenvExpand.expand(myEnv);
+
+    return {
+        user1: {
+            email: process.env.TEST_USER_MAIL,
+            name: process.env.TEST_USER_MAIL,
+            password: process.env.TEST_USER_PASSWORD,
+        },
+        user2: {
+            email: process.env.TEST_USER2_MAIL,
+            name: process.env.TEST_USER2_MAIL,
+            password: process.env.TEST_USER2_PASSWORD,
+        },
+        user3: {
+            email: process.env.TEST_USER3_MAIL,
+            name: process.env.TEST_USER3_MAIL,
+            password: process.env.TEST_USER3_PASSWORD,
+        },
+    }
+}
+
+async function waitFor(url: String, browser: Browser) {
+    var ready = false;
+    var context;
+
+    do {
+        try {
+            context = await browser.newContext();
+            const page = await context.newPage();
+            await page.waitForTimeout(500);
+            const result = await page.goto(url);
+            ready = result.status() === 200;
+        } catch(e) {
+            if( !e.message.includes("CONNECTION_REFUSED") ){
+                throw e;
+            }
+        } finally {
+            await context.close();
+        }
+    } while(!ready);
+}
+
+function startComposeService(serviceName: String){
+    console.log(`Starting ${serviceName}`);
+    execSync(`docker compose --env-file test.env  up -d ${serviceName}`);
+}
+
+function stopComposeService(serviceName: String){
+    console.log(`Stopping ${serviceName}`);
+    execSync(`docker compose --env-file test.env  stop ${serviceName}`);
+}
+
+function wipeSqlite(){
+    console.log(`Delete Vaultwarden container to wipe sqlite`);
+    execSync(`docker compose --env-file test.env stop Vaultwarden`);
+    execSync(`docker compose --env-file test.env rm -f Vaultwarden`);
+}
+
+async function wipeMariaDB(){
+    var mysql = require('mysql2/promise');
+    var ready = false;
+    var connection;
+
+    do {
+        try {
+            connection = await mysql.createConnection({
+                user: process.env.MARIADB_USER,
+                host: "127.0.0.1",
+                database: process.env.MARIADB_DATABASE,
+                password: process.env.MARIADB_PASSWORD,
+                port: process.env.MARIADB_PORT,
+            });
+
+            await connection.execute(`DROP DATABASE ${process.env.MARIADB_DATABASE}`);
+            await connection.execute(`CREATE DATABASE ${process.env.MARIADB_DATABASE}`);
+            console.log('Successfully wiped mariadb');
+            ready = true;
+        } catch (err) {
+            console.log(`Error when wiping mariadb: ${err}`);
+        } finally {
+            if( connection ){
+                connection.end();
+            }
+        }
+        await new Promise(r => setTimeout(r, 1000));
+    } while(!ready);
+}
+
+async function wipeMysqlDB(){
+    var mysql = require('mysql2/promise');
+    var ready = false;
+    var connection;
+
+    do{
+        try {
+            connection = await mysql.createConnection({
+                user: process.env.MYSQL_USER,
+                host: "127.0.0.1",
+                database: process.env.MYSQL_DATABASE,
+                password: process.env.MYSQL_PASSWORD,
+                port: process.env.MYSQL_PORT,
+            });
+
+            await connection.execute(`DROP DATABASE ${process.env.MYSQL_DATABASE}`);
+            await connection.execute(`CREATE DATABASE ${process.env.MYSQL_DATABASE}`);
+            console.log('Successfully wiped mysql');
+            ready = true;
+        } catch (err) {
+            console.log(`Error when wiping mysql: ${err}`);
+        } finally {
+            if( connection ){
+                connection.end();
+            }
+        }
+        await new Promise(r => setTimeout(r, 1000));
+    } while(!ready);
+}
+
+async function wipePostgres(){
+    const { Client } = require('pg');
+
+    const client = new Client({
+        user: process.env.POSTGRES_USER,
+        host: "127.0.0.1",
+        database: "postgres",
+        password: process.env.POSTGRES_PASSWORD,
+        port: process.env.POSTGRES_PORT,
+    });
+
+    try {
+        await client.connect();
+        await client.query(`DROP DATABASE ${process.env.POSTGRES_DB}`);
+        await client.query(`CREATE DATABASE ${process.env.POSTGRES_DB}`);
+        console.log('Successfully wiped postgres');
+    } catch (err) {
+        console.log(`Error when wiping postgres: ${err}`);
+    } finally {
+        client.end();
+    }
+}
+
+function dbConfig(testInfo: TestInfo){
+    switch(testInfo.project.name) {
+        case "postgres": return {
+            DATABASE_URL: `postgresql://${process.env.POSTGRES_USER}:${process.env.POSTGRES_PASSWORD}@127.0.0.1:${process.env.POSTGRES_PORT}/${process.env.POSTGRES_DB}`
+        }
+        case "mariadb": return {
+            DATABASE_URL: `mysql://${process.env.MARIADB_USER}:${process.env.MARIADB_PASSWORD}@127.0.0.1:${process.env.MARIADB_PORT}/${process.env.MARIADB_DATABASE}`
+        }
+        case "mysql": return {
+            DATABASE_URL: `mysql://${process.env.MYSQL_USER}:${process.env.MYSQL_PASSWORD}@127.0.0.1:${process.env.MYSQL_PORT}/${process.env.MYSQL_DATABASE}`
+        }
+        default: return { I_REALLY_WANT_VOLATILE_STORAGE: true }
+    }
+}
+
+/**
+ *  All parameters passed in `env` need to be added to the docker-compose.yml
+ **/
+async function startVaultwarden(browser: Browser, testInfo: TestInfo, env = {}, resetDB: Boolean = true) {
+    if( resetDB ){
+        switch(testInfo.project.name) {
+            case "postgres":
+                await wipePostgres();
+                break;
+            case "mariadb":
+                await wipeMariaDB();
+                break;
+            case "mysql":
+                await wipeMysqlDB();
+                break;
+            default:
+                wipeSqlite();
+        }
+    }
+
+    console.log(`Starting Vaultwarden`);
+    execSync(`docker compose --env-file test.env up -d Vaultwarden`, {
+        env: { ...env, ...dbConfig(testInfo) },
+    });
+    await waitFor("/", browser);
+    console.log(`Vaultwarden running on: ${process.env.DOMAIN}`);
+}
+
+async function stopVaultwarden() {
+    console.log(`Vaultwarden stopping`);
+    execSync(`docker compose --env-file test.env stop Vaultwarden`);
+}
+
+async function restartVaultwarden(page: Page, testInfo: TestInfo, env, resetDB: Boolean = true) {
+    stopVaultwarden();
+    return startVaultwarden(page.context().browser(), testInfo, env, resetDB);
+}
+
+export { loadEnv, waitFor, startComposeService, stopComposeService, startVaultwarden, stopVaultwarden, restartVaultwarden };
--- a/playwright/package-lock.json
+++ b/playwright/package-lock.json
--- a/playwright/package.json
+++ b/playwright/package.json
@ -0,0 +1,20 @@
+{
+    "name": "scenarios",
+    "version": "1.0.0",
+    "description": "",
+    "main": "index.js",
+    "scripts": {},
+    "keywords": [],
+    "author": "",
+    "license": "ISC",
+    "devDependencies": {
+        "@playwright/test": "^1.45.1",
+        "dotenv": "^16.4.5",
+        "dotenv-expand": "^11.0.6",
+        "maildev": "github:timshel/maildev#3.0.0-rc1"
+    },
+    "dependencies": {
+        "mysql2": "^3.10.2",
+        "pg": "^8.12.0"
+    }
+}
--- a/playwright/playwright.config.ts
+++ b/playwright/playwright.config.ts
@ -0,0 +1,96 @@
+import { defineConfig, devices } from '@playwright/test';
+import { exec } from 'node:child_process';
+
+const utils = require('./global-utils');
+
+utils.loadEnv();
+
+/**
+ * See https://playwright.dev/docs/test-configuration.
+ */
+export default defineConfig({
+    testDir: 'tests',
+    /* Run tests in files in parallel */
+    fullyParallel: false,
+
+    /* Fail the build on CI if you accidentally left test.only in the source code. */
+    forbidOnly: !!process.env.CI,
+
+    /* Retry on CI only */
+    retries: process.env.CI ? 2 : 0,
+    workers: 1,
+
+    /* Reporter to use. See https://playwright.dev/docs/test-reporters */
+    reporter: 'html',
+    timeout: 20 * 1000,
+    expect: { timeout: 10 * 1000 },
+
+    /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
+    use: {
+        /* Base URL to use in actions like `await page.goto('/')`. */
+        baseURL: process.env.DOMAIN,
+        browserName: 'firefox',
+        /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
+        trace: 'on-first-retry',
+    },
+
+    /* Configure projects for major browsers */
+    projects: [
+        {
+            name: 'mariadb-setup',
+            testMatch: 'tests/setups/db-setup.ts',
+            use: { serviceName: "Mariadb" },
+            teardown: 'mariadb-teardown',
+        },
+        {
+            name: 'mysql-setup',
+            testMatch: 'tests/setups/db-setup.ts',
+            use: { serviceName: "Mysql" },
+            teardown: 'mysql-teardown',
+        },
+        {
+            name: 'postgres-setup',
+            testMatch: 'tests/setups/db-setup.ts',
+            use: { serviceName: "Postgres" },
+            teardown: 'postgres-teardown',
+        },
+
+        {
+            name: 'mariadb',
+            testIgnore: 'tests/setups',
+            dependencies: ['mariadb-setup'],
+        },
+        {
+            name: 'mysql',
+            testIgnore: 'tests/setups',
+            dependencies: ['mysql-setup'],
+        },
+        {
+            name: 'postgres',
+            testIgnore: 'tests/setups',
+            dependencies: ['postgres-setup'],
+        },
+        {
+            name: 'sqlite',
+            testIgnore: 'tests/setups',
+        },
+
+        {
+            name: 'mariadb-teardown',
+            testMatch: 'tests/setups/db-teardown.ts',
+            use: { serviceName: "Mariadb" },
+        },
+        {
+            name: 'mysql-teardown',
+            testMatch: 'tests/setups/db-teardown.ts',
+            use: { serviceName: "Mysql" },
+        },
+        {
+            name: 'postgres-teardown',
+            testMatch: 'tests/setups/db-teardown.ts',
+            use: { serviceName: "Postgres" },
+        },
+    ],
+
+    globalSetup: require.resolve('./global-setup'),
+});
--- a/playwright/test.env
+++ b/playwright/test.env
@ -0,0 +1,71 @@
+##################################################################
+### Shared Playwright conf test file Vaultwarden and Databases ###
+##################################################################
+
+ENV=test
+COMPOSE_IGNORE_ORPHANS=True
+DOCKER_BUILDKIT=1
+VAULTWARDEN_SMTP_FROM=vaultwarden@playwright.test
+
+#####################
+# Playwright Config #
+#####################
+PW_KEEP_SERVICE_RUNNNING=${PW_KEEP_SERVICE_RUNNNING:-false}
+
+#####################
+# Maildev Config 	#
+#####################
+MAILDEV_HTTP_PORT=1081
+MAILDEV_SMTP_PORT=1026
+MAILDEV_HOST=127.0.0.1
+
+#############
+# Test user #
+#############
+TEST_USER=test
+TEST_USER_PASSWORD=Master Password
+TEST_USER_MAIL=${TEST_USER}@example.com
+
+TEST_USER2=test2
+TEST_USER2_PASSWORD=Master Password
+TEST_USER2_MAIL=${TEST_USER2}@example.com
+
+TEST_USER3=test3
+TEST_USER3_PASSWORD=Master Password
+TEST_USER3_MAIL=${TEST_USER3}@example.com
+
+######################
+# Vaultwarden Config #
+######################
+ROCKET_PORT=8003
+DOMAIN=http://127.0.0.1:${ROCKET_PORT}
+SMTP_SECURITY=off
+SMTP_PORT=${MAILDEV_SMTP_PORT}
+SMTP_FROM_NAME=Vaultwarden
+SMTP_TIMEOUT=5
+
+###########################
+# Docker MariaDb container#
+###########################
+MARIADB_PORT=3307
+MARIADB_ROOT_PASSWORD=vaultwarden
+MARIADB_USER=vaultwarden
+MARIADB_PASSWORD=vaultwarden
+MARIADB_DATABASE=vaultwarden
+
+###########################
+# Docker Mysql container#
+###########################
+MYSQL_PORT=3309
+MYSQL_ROOT_PASSWORD=vaultwarden
+MYSQL_USER=vaultwarden
+MYSQL_PASSWORD=vaultwarden
+MYSQL_DATABASE=vaultwarden
+
+############################
+# Docker Postgres container#
+############################
+POSTGRES_PORT=5433
+POSTGRES_USER=vaultwarden
+POSTGRES_PASSWORD=vaultwarden
+POSTGRES_DB=vaultwarden
--- a/playwright/tests/login.smtp.spec.ts
+++ b/playwright/tests/login.smtp.spec.ts
@ -0,0 +1,159 @@
+import { test, expect, type TestInfo } from '@playwright/test';
+import { MailDev } from 'maildev';
+
+const utils = require('../global-utils');
+import { createAccount, logUser } from './setups/user';
+
+let users = utils.loadEnv();
+
+let mailserver;
+
+test.beforeAll('Setup', async ({ browser }, testInfo: TestInfo) => {
+    mailserver = new MailDev({
+        port: process.env.MAILDEV_SMTP_PORT,
+        web: { port: process.env.MAILDEV_HTTP_PORT },
+    })
+
+    await mailserver.listen();
+
+    await utils.startVaultwarden(browser, testInfo, {
+        SMTP_HOST: process.env.MAILDEV_HOST,
+        SMTP_FROM: process.env.VAULTWARDEN_SMTP_FROM,
+    });
+});
+
+test.afterAll('Teardown', async ({}) => {
+    utils.stopVaultwarden();
+    if( mailserver ){
+        await mailserver.close();
+    }
+});
+
+test('Account creation', async ({ page }) => {
+    const emails = mailserver.iterator(users.user1.email);
+
+    await createAccount(test, page, users.user1);
+
+    const { value: created } = await emails.next();
+    expect(created.subject).toBe("Welcome");
+    expect(created.from[0]?.address).toBe(process.env.VAULTWARDEN_SMTP_FROM);
+
+    // Back to the login page
+    await expect(page).toHaveTitle('Vaultwarden Web');
+    await expect(page.getByTestId("toast-message")).toHaveText(/Your new account has been created/);
+    await page.getByRole('button', { name: 'Continue' }).click();
+
+    // Unlock page
+    await page.getByLabel('Master password').fill(users.user1.password);
+    await page.getByRole('button', { name: 'Log in with master password' }).click();
+
+    // We are now in the default vault page
+    await expect(page).toHaveTitle(/Vaults/);
+
+    const { value: logged } = await emails.next();
+    expect(logged.subject).toBe("New Device Logged In From Firefox");
+    expect(logged.to[0]?.address).toBe(process.env.TEST_USER_MAIL);
+    expect(logged.from[0]?.address).toBe(process.env.VAULTWARDEN_SMTP_FROM);
+
+    emails.return();
+});
+
+test('Login', async ({ context, page }) => {
+    const emails = mailserver.iterator(users.user1.email);
+
+    await logUser(test, page, users.user1);
+
+    await test.step('new device email', async () => {
+        const { value: logged } = await emails.next();
+        expect(logged.subject).toBe("New Device Logged In From Firefox");
+        expect(logged.from[0]?.address).toBe(process.env.VAULTWARDEN_SMTP_FROM);
+    });
+
+    await test.step('verify email', async () => {
+        await page.getByText('Verify your account\'s email').click();
+        await expect(page.getByTestId("toast-message")).toHaveText(/Check your email inbox for a verification link/);
+        await page.locator('#toast-container').getByRole('button').click();
+        await expect(page.getByTestId("toast-message")).toHaveCount(0);
+
+        const { value: verify } = await emails.next();
+        expect(verify.subject).toBe("Verify Your Email");
+        expect(verify.from[0]?.address).toBe(process.env.VAULTWARDEN_SMTP_FROM);
+
+        const page2 = await context.newPage();
+        await page2.setContent(verify.html);
+        const link = await page2.getByTestId("verify").getAttribute("href");
+        await page2.close();
+
+        await page.goto(link);
+        await expect(page.getByTestId("toast-message")).toHaveText("Account email verified");
+    });
+
+    emails.return();
+});
+
+test('Activaite 2fa', async ({ context, page }) => {
+    const emails = mailserver.buffer(users.user1.email);
+
+    await logUser(test, page, users.user1);
+
+    await page.getByRole('button', { name: users.user1.name }).click();
+    await page.getByRole('menuitem', { name: 'Account settings' }).click();
+    await page.getByLabel('Security').click();
+    await page.getByRole('link', { name: 'Two-step login' }).click();
+    await page.locator('li').filter({ hasText: 'Email Verification codes will' }).getByRole('button').click();
+    await page.getByLabel('Master password (required)').fill(users.user1.password);
+    await page.getByRole('button', { name: 'Continue' }).click();
+    await page.getByRole('button', { name: 'Send email' }).click();
+
+    const codeMail = await emails.next((mail) => mail.subject === "Vaultwarden Login Verification Code");
+    const page2 = await context.newPage();
+    await page2.setContent(codeMail.html);
+    const code = await page2.getByTestId("2fa").innerText();
+    await page2.close();
+
+    await page.getByLabel('2. Enter the resulting 6').fill(code);
+    await page.getByRole('button', { name: 'Turn on' }).click();
+    await page.getByRole('heading', { name: 'Turned on', exact: true });
+
+    emails.close();
+});
+
+test('2fa', async ({ context, page }) => {
+    const emails = mailserver.buffer(users.user1.email);
+
+    await test.step('login', async () => {
+        await page.goto('/');
+
+        await page.getByLabel(/Email address/).fill(users.user1.email);
+        await page.getByRole('button', { name: 'Continue' }).click();
+        await page.getByLabel('Master password').fill(users.user1.password);
+        await page.getByRole('button', { name: 'Log in with master password' }).click();
+
+        const codeMail = await emails.next((mail) => mail.subject === "Vaultwarden Login Verification Code");
+        const page2 = await context.newPage();
+        await page2.setContent(codeMail.html);
+        const code = await page2.getByTestId("2fa").innerText();
+        await page2.close();
+
+        await page.getByLabel('Verification code').fill(code);
+        await page.getByRole('button', { name: 'Continue' }).click();
+
+        await expect(page).toHaveTitle(/Vaults/);
+    })
+
+    await test.step('disable', async () => {
+        await page.getByRole('button', { name: 'Test' }).click();
+        await page.getByRole('menuitem', { name: 'Account settings' }).click();
+        await page.getByLabel('Security').click();
+        await page.getByRole('link', { name: 'Two-step login' }).click();
+        await page.locator('li').filter({ hasText: 'Email Turned on Verification' }).getByRole('button').click();
+        await page.getByLabel('Master password (required)').click();
+        await page.getByLabel('Master password (required)').fill(users.user1.password);
+        await page.getByRole('button', { name: 'Continue' }).click();
+        await page.getByRole('button', { name: 'Turn off' }).click();
+        await page.getByRole('button', { name: 'Yes' }).click();
+        await expect(page.getByTestId("toast-message")).toHaveText(/Two-step login provider turned off/);
+    });
+
+    emails.close();
+});
--- a/playwright/tests/login.spec.ts
+++ b/playwright/tests/login.spec.ts
@ -0,0 +1,32 @@
+import { test, expect, type Page, type TestInfo } from '@playwright/test';
+
+import * as utils from "../global-utils";
+import { createAccount, logUser } from './setups/user';
+
+let users = utils.loadEnv();
+
+test.beforeAll('Setup', async ({ browser }, testInfo: TestInfo) => {
+    await utils.startVaultwarden(browser, testInfo, {});
+});
+
+test.afterAll('Teardown', async ({}, testInfo: TestInfo) => {
+    utils.stopVaultwarden(testInfo);
+});
+
+test('Account creation', async ({ page }) => {
+    // Landing page
+    await createAccount(test, page, users.user1);
+
+    await page.getByRole('button', { name: 'Continue' }).click();
+
+    // Unlock page
+    await page.getByLabel('Master password').fill(users.user1.password);
+    await page.getByRole('button', { name: 'Log in with master password' }).click();
+
+    // We are now in the default vault page
+    await expect(page).toHaveTitle(/Vaults/);
+});
+
+test('Master password login', async ({ page }) => {
+    await logUser(test, page, users.user1);
+});
--- a/playwright/tests/organization.spec.ts
+++ b/playwright/tests/organization.spec.ts
@ -0,0 +1,167 @@
+import { test, expect, type TestInfo } from '@playwright/test';
+import { MailDev } from 'maildev';
+
+import * as utils from "../global-utils";
+import { createAccount, logUser } from './setups/user';
+
+let users = utils.loadEnv();
+
+let mailserver, user1Mails, user2Mails, user3Mails;
+
+test.beforeAll('Setup', async ({ browser }, testInfo: TestInfo) => {
+    mailserver = new MailDev({
+        port: process.env.MAILDEV_SMTP_PORT,
+        web: { port: process.env.MAILDEV_HTTP_PORT },
+    })
+
+    await mailserver.listen();
+
+    await utils.startVaultwarden(browser, testInfo, {
+        SMTP_HOST: process.env.MAILDEV_HOST,
+        SMTP_FROM: process.env.VAULTWARDEN_SMTP_FROM,
+    });
+
+    user1Mails = mailserver.iterator(users.user1.email);
+    user2Mails = mailserver.iterator(users.user2.email);
+    user3Mails = mailserver.iterator(users.user3.email);
+});
+
+test.afterAll('Teardown', async ({}, testInfo: TestInfo) => {
+    utils.stopVaultwarden(testInfo);
+    if( mailserver ){
+        await mailserver.close();
+
+        [user1Mails, user2Mails, user3Mails].map((mails) => {
+            if(mails){
+                mails.return();
+            }
+        });
+    }
+});
+
+test('Create user3', async ({ page }) => {
+    await createAccount(test, page, users.user3, user3Mails);
+});
+
+test('Invite users', async ({ page }) => {
+    await createAccount(test, page, users.user1, user1Mails);
+    await logUser(test, page, users.user1, user1Mails);
+
+    await test.step('Create Org', async () => {
+        await page.getByRole('link', { name: 'New organization' }).click();
+        await page.getByLabel('Organization name (required)').fill('Test');
+        await page.getByRole('button', { name: 'Submit' }).click();
+        await page.locator('div').filter({ hasText: 'Members' }).nth(2).click();
+    });
+
+    await test.step('Invite user2', async () => {
+        await page.getByRole('button', { name: 'Invite member' }).click();
+        await page.getByLabel('Email (required)').fill(users.user2.email);
+        await page.getByRole('tab', { name: 'Collections' }).click();
+        await page.locator('label').filter({ hasText: 'Grant access to all current' }).click();
+        await page.getByRole('button', { name: 'Save' }).click();
+        await expect(page.getByTestId("toast-message")).toHaveText('User(s) invited');
+    });
+
+    await test.step('Invite user3', async () => {
+        await page.getByRole('button', { name: 'Invite member' }).click();
+        await page.getByLabel('Email (required)').fill(users.user3.email);
+        await page.getByRole('tab', { name: 'Collections' }).click();
+        await page.locator('label').filter({ hasText: 'Grant access to all current' }).click();
+        await page.getByRole('button', { name: 'Save' }).click();
+        await expect(page.getByTestId("toast-message")).toHaveText('User(s) invited');
+    });
+});
+
+test('invited with new account', async ({ page }) => {
+    const { value: invited } = await user2Mails.next();
+    expect(invited.subject).toContain("Join Test")
+
+    await test.step('Create account', async () => {
+        await page.setContent(invited.html);
+        const link = await page.getByTestId("invite").getAttribute("href");
+        await page.goto(link);
+        await expect(page).toHaveTitle(/Create account | Vaultwarden Web/);
+
+        await page.getByLabel('Name').fill(users.user2.name);
+        await page.getByLabel('Master password\n   (required)', { exact: true }).fill(users.user2.password);
+        await page.getByLabel('Re-type master password').fill(users.user2.password);
+        await page.getByRole('button', { name: 'Create account' }).click();
+
+        // Back to the login page
+        await expect(page).toHaveTitle('Vaultwarden Web');
+        await expect(page.getByTestId("toast-message")).toHaveText(/Your new account has been created/);
+
+        const { value: welcome } = await user2Mails.next();
+        expect(welcome.subject).toContain("Welcome")
+    });
+
+    await test.step('Login', async () => {
+        await page.getByLabel(/Email address/).fill(users.user2.email);
+        await page.getByRole('button', { name: 'Continue' }).click();
+
+        // Unlock page
+        await page.getByLabel('Master password').fill(users.user2.password);
+        await page.getByRole('button', { name: 'Log in with master password' }).click();
+
+        // We are now in the default vault page
+        await expect(page).toHaveTitle(/Vaults/);
+        await expect(page.getByTestId("toast-title")).toHaveText("Invitation accepted");
+
+        const { value: logged } = await user2Mails.next();
+        expect(logged.subject).toContain("New Device Logged");
+    });
+
+    const { value: accepted } = await user1Mails.next();
+    expect(accepted.subject).toContain("Invitation to Test accepted")
+});
+
+test('invited with existing account', async ({ page }) => {
+    const { value: invited } = await user3Mails.next();
+    expect(invited.subject).toContain("Join Test")
+
+    await page.setContent(invited.html);
+    const link = await page.getByTestId("invite").getAttribute("href");
+
+    await page.goto(link);
+
+    // We should be on login page with email prefilled
+    await expect(page).toHaveTitle(/Vaultwarden Web/);
+    await page.getByRole('button', { name: 'Continue' }).click();
+
+    // Unlock page
+    await page.getByLabel('Master password').fill(users.user3.password);
+    await page.getByRole('button', { name: 'Log in with master password' }).click();
+
+    // We are now in the default vault page
+    await expect(page).toHaveTitle(/Vaults/);
+    await expect(page.getByTestId("toast-title")).toHaveText("Invitation accepted");
+
+    const { value: logged } = await user3Mails.next();
+    expect(logged.subject).toContain("New Device Logged")
+
+    const { value: accepted } = await user1Mails.next();
+    expect(accepted.subject).toContain("Invitation to Test accepted")
+});
+
+test('Confirm invited user', async ({ page }) => {
+    await logUser(test, page, users.user1, user1Mails);
+    await page.getByLabel('Switch products').click();
+    await page.getByRole('link', { name: ' Admin Console' }).click();
+    await page.getByLabel('Members').click();
+
+    await test.step('Accept user2', async () => {
+        await page.getByRole('row', { name: users.user2.name }).getByLabel('Options').click();
+        await page.getByRole('menuitem', { name: 'Confirm' }).click();
+        await page.getByRole('button', { name: 'Confirm' }).click();
+        await expect(page.getByTestId("toast-message")).toHaveText(/confirmed/);
+
+        const { value: logged } = await user2Mails.next();
+        expect(logged.subject).toContain("Invitation to Test confirmed");
+    });
+});
+
+test('Organization is visible', async ({ page }) => {
+    await logUser(test, page, users.user2, user2Mails);
+    await page.getByLabel('vault: Test').click();
+});
--- a/playwright/tests/setups/db-setup.ts
+++ b/playwright/tests/setups/db-setup.ts
@ -0,0 +1,7 @@
+import { test } from './db-test';
+
+const utils = require('../../global-utils');
+
+test('DB start', async ({ serviceName }) => {
+	utils.startComposeService(serviceName);
+});
--- a/playwright/tests/setups/db-teardown.ts
+++ b/playwright/tests/setups/db-teardown.ts
@ -0,0 +1,11 @@
+import { test } from './db-test';
+
+const utils = require('../../global-utils');
+
+utils.loadEnv();
+
+test('DB teardown ?', async ({ serviceName }) => {
+    if( process.env.PW_KEEP_SERVICE_RUNNNING !== "true" ) {
+        utils.stopComposeService(serviceName);
+    }
+});
--- a/playwright/tests/setups/db-test.ts
+++ b/playwright/tests/setups/db-test.ts
@ -0,0 +1,9 @@
+import { test as base } from '@playwright/test';
+
+export type TestOptions = {
+  serviceName: string;
+};
+
+export const test = base.extend<TestOptions>({
+  serviceName: ['', { option: true }],
+});
--- a/playwright/tests/setups/user.ts
+++ b/playwright/tests/setups/user.ts
@ -0,0 +1,47 @@
+import { expect, type Browser,Page } from '@playwright/test';
+
+export async function createAccount(test, page: Page, user: { email: string, name: string, password: string }, emails) {
+    await test.step('Create user', async () => {
+        // Landing page
+        await page.goto('/');
+        await page.getByRole('link', { name: 'Create account' }).click();
+
+        // Back to Vault create account
+        await expect(page).toHaveTitle(/Create account | Vaultwarden Web/);
+        await page.getByLabel(/Email address/).fill(user.email);
+        await page.getByLabel('Name').fill(user.name);
+        await page.getByLabel('Master password\n   (required)', { exact: true }).fill(user.password);
+        await page.getByLabel('Re-type master password').fill(user.password);
+        await page.getByRole('button', { name: 'Create account' }).click();
+
+        // Back to the login page
+        await expect(page).toHaveTitle('Vaultwarden Web');
+        await expect(page.getByTestId("toast-message")).toHaveText(/Your new account has been created/);
+
+        if( emails ){
+            const { value: welcome } = await emails.next();
+            expect(welcome.subject).toContain("Welcome");
+        }
+    });
+}
+
+export async function logUser(test, page: Page, user: { email: string, password: string }, emails) {
+    await test.step('Log user', async () => {
+        // Landing page
+        await page.goto('/');
+        await page.getByLabel(/Email address/).fill(user.email);
+        await page.getByRole('button', { name: 'Continue' }).click();
+
+        // Unlock page
+        await page.getByLabel('Master password').fill(user.password);
+        await page.getByRole('button', { name: 'Log in with master password' }).click();
+
+        // We are now in the default vault page
+        await expect(page).toHaveTitle(/Vaults/);
+
+        if( emails ){
+            const { value: logged } = await emails.next();
+            expect(logged.subject).toContain("New Device Logged");
+        }
+    });
+}
--- a/src/static/templates/email/send_org_invite.html.hbs
+++ b/src/static/templates/email/send_org_invite.html.hbs
@ -9,7 +9,7 @@ Join {{{org_name}}}
   </tr>
   <tr style="margin: 0; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; box-sizing: border-box; font-size: 16px; color: #333; line-height: 25px; -webkit-font-smoothing: antialiased; -webkit-text-size-adjust: none;">
      <td class="content-block" style="font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; box-sizing: border-box; font-size: 16px; color: #333; line-height: 25px; margin: 0; -webkit-font-smoothing: antialiased; padding: 0 0 10px; -webkit-text-size-adjust: none; text-align: center;" valign="top" align="center">
-         <a href="{{url}}"
+         <a data-testid="invite" href="{{url}}"
            clicktracking=off target="_blank" style="color: #ffffff; text-decoration: none; text-align: center; cursor: pointer; display: inline-block; border-radius: 5px; background-color: #3c8dbc; border-color: #3c8dbc; border-style: solid; border-width: 10px 20px; margin: 0; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; box-sizing: border-box; font-size: 16px; line-height: 25px; -webkit-font-smoothing: antialiased; -webkit-text-size-adjust: none;">
         Join Organization Now
         </a>
--- a/src/static/templates/email/twofactor_email.html.hbs
+++ b/src/static/templates/email/twofactor_email.html.hbs
@ -4,7 +4,7 @@ Vaultwarden Login Verification Code
 <table width="100%" cellpadding="0" cellspacing="0" style="margin: 0; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; box-sizing: border-box; font-size: 16px; color: #333; line-height: 25px; -webkit-font-smoothing: antialiased; -webkit-text-size-adjust: none;">
    <tr style="margin: 0; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; box-sizing: border-box; font-size: 16px; color: #333; line-height: 25px; -webkit-font-smoothing: antialiased; -webkit-text-size-adjust: none;">
        <td class="content-block" style="font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; box-sizing: border-box; font-size: 16px; color: #333; line-height: 25px; margin: 0; -webkit-font-smoothing: antialiased; padding: 0 0 10px; -webkit-text-size-adjust: none;" valign="top">
-            Your two-step verification code is: <b>{{token}}</b>
+            Your two-step verification code is: <b data-testid="2fa">{{token}}</b>
        </td>
    </tr>
    <tr style="margin: 0; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; box-sizing: border-box; font-size: 16px; color: #333; line-height: 25px; -webkit-font-smoothing: antialiased; -webkit-text-size-adjust: none;">
--- a/src/static/templates/email/verify_email.html.hbs
+++ b/src/static/templates/email/verify_email.html.hbs
@ -9,7 +9,7 @@ Verify Your Email
   </tr>
   <tr style="margin: 0; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; box-sizing: border-box; font-size: 16px; color: #333; line-height: 25px; -webkit-font-smoothing: antialiased; -webkit-text-size-adjust: none;">
      <td class="content-block" style="font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; box-sizing: border-box; font-size: 16px; color: #333; line-height: 25px; margin: 0; -webkit-font-smoothing: antialiased; padding: 0 0 10px; -webkit-text-size-adjust: none; text-align: center;" valign="top" align="center">
-         <a href="{{url}}/#/verify-email/?userId={{user_id}}&token={{token}}"
+         <a data-testid="verify" href="{{url}}/#/verify-email/?userId={{user_id}}&token={{token}}"
            clicktracking=off target="_blank" style="color: #ffffff; text-decoration: none; text-align: center; cursor: pointer; display: inline-block; border-radius: 5px; background-color: #3c8dbc; border-color: #3c8dbc; border-style: solid; border-width: 10px 20px; margin: 0; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; box-sizing: border-box; font-size: 16px; line-height: 25px; -webkit-font-smoothing: antialiased; -webkit-text-size-adjust: none;">
         Verify Email Address Now
         </a>
@ -21,4 +21,4 @@ Verify Your Email
      </td>
   </tr>
 </table>
-{{> email/email_footer }}
+{{> email/email_footer }}