With this commit I have changed several components to be more efficient.
This can be less llvm-lines generated or less `clone()` calls.
### Config
- Re-ordered the `make_config` macro to be more efficient
- Created a custom Deserializer for `ConfigBuilder` less code and more efficient
- Use struct's for the `prepare_json` function instead of generating a custom JSON object.
This generates less code and is more efficient.
- Updated the `get_support_string` function to handle the masking differently.
This generates less code and also was able to remove some sub-macro-calls
### Error
- Added an extra new call to prevent duplicate Strings in generated macro code.
This generated less llvm-lines and seems to be more efficient.
- Created a custom Serializer for `ApiError` and `CompactApiError`
This makes that struct smaller in size, so better for memory, but also less llvm-lines.
### General
- Removed `once_lock` and replace it all with Rust's std LazyLock
- Added and fixed some Clippy lints which reduced `clone()` calls for example.
- Updated build profiles for more efficiency
Also added a new profile specifically for CI, which should decrease the build check
- Updated several GitHub Workflows for better security and use the new `ci` build profile
- Updated to Rust v1.90.0 which uses a new linker `rust-lld` which should help in faster building
- Updated the Cargo.toml for all crates to better use the `workspace` variables
- Added a `typos` Workflow and Pre-Commit, which should help in detecting spell error's.
Also fixed a few found by it.
Signed-off-by: BlackDex <black.dex@gmail.com>
- Updated crates
- Switched to rustls instead of native-tls
Some dependency were already using rustls by default or without option.
By removing native-tls we also have just one way of working here.
Updated favicon fetching which now is able to fetch more icons.
- Use rustls instead of native-tls
This seems to work better, probably because of tls sniffing
- Use different user-agent and added several other headers
- Added SVG support. SVG Images will be sanitized first before stored or presented.
Also, a special CSP for images will be sent to prevent scripts etc.. from SVG images.
Signed-off-by: BlackDex <black.dex@gmail.com>
* Abstract file access through Apache OpenDAL
* Add AWS S3 support via OpenDAL for data files
* PR improvements
* Additional PR improvements
* Config setting comments for local/remote data locations
- Updated web-vault to v2025.5.0
- Updated Rust to v1.87.0
- Updated all the crates
- Replaced yubico with yubico_ng
- Fixed several new (nightly) clippy lints
Signed-off-by: BlackDex <black.dex@gmail.com>
* Fix icon redirect not working on desktop
We also need to exclude the header in case we do an external_icon call.
Fixes#5535
Signed-off-by: BlackDex <black.dex@gmail.com>
* Add informational comments to the icon_external function
Signed-off-by: BlackDex <black.dex@gmail.com>
* Fix spelling/grammar
Signed-off-by: BlackDex <black.dex@gmail.com>
---------
Signed-off-by: BlackDex <black.dex@gmail.com>
* Add manage role for collections and groups
This commit will add the manage role/column to collections and groups.
We need this to allow users part of a collection either directly or via groups to be able to delete ciphers.
Without this, they are only able to either edit or view them when using new clients, since these check the manage role.
Still trying to keep it compatible with previous versions and able to revert to an older Vaultwarden version and the `access_all` feature of the older installations.
In a future version we should really check and fix these rights and create some kind of migration step to also remove the `access_all` feature and convert that to a `manage` option.
But this commit at least creates the base for this already.
This should resolve#5367
Signed-off-by: BlackDex <black.dex@gmail.com>
* Fix an issue with access_all
If owners or admins do not have the `access_all` flag set, in case they do not want to see all collection on the password manager view, they didn't see any collections at all anymore.
This should fix that they are still able to view all the collections and have access to it.
Signed-off-by: BlackDex <black.dex@gmail.com>
---------
Signed-off-by: BlackDex <black.dex@gmail.com>
* Update Rust and crates
- Updated Rust to v1.83.0
- Updated MSRV to v1.82.0 (Needed for html5gum crate)
- Updated icon fetching code to match new html5gum version
- Updated workflows
- Enabled edition 2024 clippy lints
Nightly reports some clippy hints, but that would be too much to change in this PR i think.
Signed-off-by: BlackDex <black.dex@gmail.com>
* Some additional updates
- Patch fern to allow syslog-7 feature
- Fixed diesel logger which was broken because of the sqlite backup feature
Refactored the sqlite backup because of this
- Added a build workflow test to include the query_logger feature
Signed-off-by: BlackDex <black.dex@gmail.com>
* Also patch yubico-rs and latest updates
Signed-off-by: BlackDex <black.dex@gmail.com>
---------
Signed-off-by: BlackDex <black.dex@gmail.com>
* Add extra linting
Added extra linting for some code styles.
Also added the Rust Edition 2024 lints.
Closes#4974
Signed-off-by: BlackDex <black.dex@gmail.com>
* Adjusted according to comments
Signed-off-by: BlackDex <black.dex@gmail.com>
---------
Signed-off-by: BlackDex <black.dex@gmail.com>
- Updated Rust to v1.77.0
- Updated several crates
The `reqwest` update included `trust-dns` > `hickory-dns` changes.
Also, `reqwest` v0.12 is not working correctly for us, that is something to investigate.
- Fixed a new clippy warning
Some optimizations in regards to downloading Favicon's.
I also encounterd some issues with accessing some sites where the
connection got dropped or closed early. This seems a reqwest/hyper
thingy, https://github.com/hyperium/hyper/issues/2136. This is now also
fixed.
General:
- Decreased struct size
- Decreased memory allocations
- Optimized tokenizer a bit more to only emit tags when all attributes are there and are valid.
reqwest/hyper connection issue:
The following changes helped solve the connection issues to some sites.
The endresult is that some icons are now able to be downloaded always instead of sometimes.
- Enabled some extra reqwest features, `deflate` and `native-tls-alpn`
(Which do not bring in any extra crates since other crates already enabled them, but they were not active for Vaultwarden it self)
- Configured reqwest to have a max amount of idle pool connections per host
- Configured reqwest to timeout the idle connections in 10 seconds
During setting the latest commit hash for Rocket and updating all the
other crates, there were some messages regarding the usage of `String`
for the Rocket endpoint function calls. I acted upon this message and
changed all `String` types to `&str` and modified the code where needed.
This ended up in less alloc calls, and probably also a bit less memory usage.
- Updated all the crates and commit hashes
- Modified all `String` to `&str` where applicable
When an icon will not be downloaded due to matching a configured
blacklist, ensure that the log message indicates the type of blacklist
that was matched.
- Several cleanups and code optimizations for Emergency Access
- Fixed a race-condition regarding jobs for Emergency Access
- Some other small changes like `allow(clippy::)` removals
Fixes#2925
- Several cleanups and code optimizations for Emergency Access
- Fixed a race-condition regarding jobs for Emergency Access
- Some other small changes like `allow(clippy::)` removals
Fixes#2925
- Several cleanups and code optimizations for Emergency Access
- Fixed a race-condition regarding jobs for Emergency Access
- Some other small changes like `allow(clippy::)` removals
Fixes#2925
- Several cleanups and code optimizations for Emergency Access
- Fixed a race-condition regarding jobs for Emergency Access
- Some other small changes like `allow(clippy::)` removals
Fixes#2925
- Several cleanups and code optimizations for Emergency Access
- Fixed a race-condition regarding jobs for Emergency Access
- Some other small changes like `allow(clippy::)` removals
Fixes#2925
When a icon blacklist regex was configured to not check for a domain, it
still did a DNS lookup first. This could cause a DNS leakage for these
regex blocked domains.
This PR resolves this issue by first checking the regex, and afterwards
the other checks.
Fixes#2909
When using anything else but the `internal` icon service it would
trigger an CSP block because the redirects were not allowed.
This PR fixes#2623 by dynamically adding the needed CSP strings.
This should also work with custom services.
For Google i needed to add an extra check because that does a redirect
it self to there gstatic.com domain.
A bit inspired by @paolobarbolini from this commit at lettre https://github.com/lettre/lettre/pull/784 .
I added a few more clippy lints here, and fixed the resulted issues.
Overall i think this could help in preventing future issues, and maybe
even peformance problems. It also makes some code a bit more clear.
We could always add more if we want to, i left a few out which i think
arn't that huge of an issue. Some like the `unused_async` are nice,
which resulted in a few `async` removals.
Some others are maybe a bit more estatic, like `string_to_string`, but i
think it looks better to use `clone` in those cases instead of `to_string` while they already are a string.
- Updated dependencies (html5gum for favicon downloading)
* Also openssl, time, jsonwebtoken and r2d2
- Small optimizations on downloading favicons.
It now only emits tokens/tags which needs to be parsed, all others are
being skipped. This prevents unneeded items within the for-loop being
parsed.
This is a rather large PR which updates the async branch to have all the
database methods as an async fn.
Some iter/map logic needed to be changed to a stream::iter().then(), but
besides that most changes were just adding async/await where needed.
Favicon:
- Replaced HTML tokenizer, much faster now.
- Caching the domain blacklist function.
- Almost all functions are async now.
- Fixed bug on minimizing data to parse
- Changed maximum icon download size to 5MB to match Bitwarden
- Added `apple-touch-icon.png` as a second fallback besides `favicon.ico`
SMTP:
- Deprecated SMTP_SSL and SMTP_EXPLICIT_TLS, replaced with SMTP_SECURITY
Misc:
- Fixed issue when `resolv.conf` contains errors and trust-dns panics (Fixes#2283)
- Updated Javscript and CSS files for admin interface
- Fixed an issue with the /admin interface which did not cleared the login cookie correctly
- Prevent websocket notifications during org import, this caused a lot of traffic, and slowed down the import.
This is also the same as Bitwarden which does not trigger this refresh via websockets.
Rust:
- Updated to use v1.59
- Use the new `strip` option and enabled to strip `debuginfo`
- Enabled `lto` with `thin`
- Removed the strip RUN from the alpine armv7, this is now done automatically
This is a rather large PR which updates the async branch to have all the
database methods as an async fn.
Some iter/map logic needed to be changed to a stream::iter().then(), but
besides that most changes were just adding async/await where needed.
The default code is 307 (temporary) to make it easier to test different icon
services, but once a service has been decided on, users should ideally switch
to using permanent redirects for cacheability.
Browsers are rather smart, but also dumb. This uses the `Expires` header
alongside `cache-control` to better prompt the browser to actually
cache.
Unfortunately, firefox still tries to "race" its own cache, in an
attempt to respond to requests faster, so still ends up making a bunch
of requests which could have been cached. Doesn't appear there's any way
around this.
If an external icon service is configured, icon requests return an HTTP
redirect to the corresponding icon at the external service.
An external service may be useful for various reasons, such as if:
* The Vaultwarden instance has no external network connectivity.
* The Vaultwarden instance has trouble handling large bursts of icon requests.
* There are concerns that an attacker may probe the instance to try to detect
whether icons for certain sites have been cached, which would suggest that
the instance contains entries for those sites.
* The external icon service does a better job of providing icons than the
built-in fetcher.
- Unicode websites could break (www.post.japanpost.jp for example).
regex would fail because it was missing the unicode-perl feature.
- Be less verbose in logging with icon downloads
- Removed duplicate info/error messages
- Added err_silent! macro to help with the less verbose error/info messages.
BlackDex
Stefan Melmuk
mountdisk
Chase Douglas
chuangjinglu
Daniel García
tuhana
Kevin P. Fleming
Jeremy Lin
Jake Howard