public/kms
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat: Utimaco General Purpose HSMs support (#367)

Browse source 
* utimaco initial

* doc on simulator

* doc on simulator

* doc: using port forwarding

* more doc

* test utimaco lib

* utimaco works - before refacto

* strated refacto

* Refacto and doc of base_hsm

* refactored utimaco in base hsm

* documentation

* support file

* proteccio drivers

* remove aider

* removed aider

* documentation

* documentation

* debugging

* fixed dangling pointers issues

* test refactoring

* working on documentation

* more doc rework

* database

* configuration

* doc more fixes

* more doc fixes

* aes encrypt hsm fixing

* syn encryption fix

* HSM doc fixes

* done with Proteccio

* enabled utimaco

* fixed utimaco

* more doc

* fmt

* changelog

* fixed sym encrypt enum serialization

* crates updates

* documentation

* chore: PR review

* cleanup

* fix: reuse cargo deps from root

---------

Co-authored-by: Manuthor <manu.coste@gmail.com>
This commit is contained in:
Bruno Grieder 2025-02-03 21:53:10 +01:00 • committed by GitHub
parent 5f9f5d0cbd
commit 48d94db4ab
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
69 changed files with 4734 additions and 1732 deletions
Download patch file Download diff file Expand all files Collapse all files

1
.gitignore vendored
View file

@ -19,3 +19,4 @@ node_modules/
# this directory may contain sqlite data when the KMS is launched locally
**/cosmian-kms/sqlite-data*
run.sh
.aider/

15
.markdown-link-check.json
View file

@ -15,6 +15,21 @@
{
"pattern": "kms.my_domain.com"
},
{
"pattern": "jwt.io"
},
{
"pattern": "www.mysql.com"
},
{
"pattern": "get--export"
},
{
"pattern": "not-possible"
},
{
"pattern": "wrapped-by-an-hsm-key"
},
{
"pattern": "mailto:"
}

20
.pre-commit-config.yaml
View file

@ -7,14 +7,14 @@
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
exclude: documentation/pandoc|documentation/overrides|crate/server/src/tests/test_utils.rs|.pre-commit-config.yaml|crate/server/src/routes/google_cse/jwt.rs|crate/server/src/routes/google_cse/python/openssl|documentation/docs/google_cse|crate/pkcs11/sys|documentation/docs/drawings|test_data|documentation/docs/benchmarks.md
exclude: documentation/pandoc|documentation/overrides|crate/server/src/tests/test_utils.rs|.pre-commit-config.yaml|crate/server/src/routes/google_cse/jwt.rs|crate/server/src/routes/google_cse/python/openssl|documentation/docs/google_cse|crate/pkcs11/sys|documentation/docs/drawings|test_data|documentation/docs/benchmarks.md|crate/hsm/proteccio/driver
repos:
- repo: https://github.com/compilerla/conventional-pre-commit
rev: v3.4.0
hooks:
- id: conventional-pre-commit
stages: [commit-msg]
args: [] # optional: list of Conventional Commits types to allow e.g. [feat, fix, ci, chore, test]
stages: [ commit-msg ]
args: [ ] # optional: list of Conventional Commits types to allow e.g. [feat, fix, ci, chore, test]
- repo: https://github.com/igorshubovych/markdownlint-cli
rev: v0.42.0
@ -32,20 +32,20 @@ repos:
rev: v3.12.2
hooks:
- id: markdown-link-check
args: [-q, --config, .markdown-link-check.json]
args: [ -q, --config, .markdown-link-check.json ]
- repo: https://github.com/jumanjihouse/pre-commit-hook-yamlfmt
rev: 0.2.3
hooks:
- id: yamlfmt
args: [--mapping, "2", --sequence, "4", --offset, "2"]
args: [ --mapping, "2", --sequence, "4", --offset, "2" ]
exclude: ansible
- repo: https://github.com/crate-ci/typos
rev: v1.25.0
hooks:
- id: typos
exclude: documentation/docs/images/google_cse.drawio.svg|crate/test_server/src/test_jwt.rs|crate/pkcs11/documentation/veracrypt_ckms.svg|crate/server/src/tests/google_cse/|documentation/docs/pkcs11/images|crate/server/resources|documentation/docs/algorithms.md|crate/server/src/tests/certificates/chain/root/ca/|documentation/docs/pki/smime.md|documentation/docs/single_server_mode.md
exclude: documentation/docs/images/google_cse.drawio.svg|crate/test_server/src/test_jwt.rs|crate/pkcs11/documentation/veracrypt_ckms.svg|crate/server/src/tests/google_cse/|documentation/docs/pkcs11/images|crate/server/resources|documentation/docs/algorithms.md|crate/server/src/tests/certificates/chain/root/ca/|documentation/docs/pki/smime.md|documentation/docs/hsms/proteccio.md
- repo: https://github.com/Lucas-C/pre-commit-hooks
rev: v1.5.5
@ -99,7 +99,7 @@ repos:
- id: fix-byte-order-marker
- id: fix-encoding-pragma
- id: mixed-line-ending
args: [--fix=lf]
args: [ --fix=lf ]
- id: name-tests-test
- id: requirements-txt-fixer
- id: sort-simple-yaml
@ -111,7 +111,7 @@ repos:
hooks:
- id: black
# avoid clash with `double-quote-string-fixer`
args: [--skip-string-normalization]
args: [ --skip-string-normalization ]
- repo: https://github.com/Cosmian/git-hooks.git
rev: v1.0.32
@ -124,7 +124,7 @@ repos:
- id: cargo-build-kms
- id: docker-compose-up
- id: cargo-test
args: [--, --skip, test_wrap_auth, --skip, google_cse, --skip, hsm]
args: [ --, --skip, test_wrap_auth, --skip, google_cse, --skip, hsm ]
- id: clippy-autofix-unreachable-pub
- id: clippy-autofix-all-targets-all-features
- id: clippy-autofix-all-targets
@ -137,4 +137,4 @@ repos:
rev: 0.16.1
hooks:
- id: cargo-deny
args: [--all-features, check]
args: [ --all-features, check ]

11
.vscode/settings.json vendored
View file

@ -61,15 +61,16 @@
"[html]": {
"editor.formatOnSave": false
},
"python.analysis.typeCheckingMode": "basic",
"rust-analyzer.check.command": "clippy",
"[python]": {
"editor.defaultFormatter": "ms-python.autopep8"
},
"[rust]": {
"editor.defaultFormatter": "rust-lang.rust-analyzer",
"editor.formatOnSave": true,
"editor.formatOnSaveMode": "file"
},
"rust-analyzer.showUnlinkedFileNotification": false
"rust-analyzer.showUnlinkedFileNotification": false,
"rust-analyzer.cargo.features": [
"utimaco",
"proteccio"
],
"editor.formatOnSave": true
}

17
CHANGELOG.md
View file

@ -2,6 +2,23 @@
All notable changes to this project will be documented in this file.
## [4.22.0] -
### 📚 Documentation
- Clarified installation documentation
- Improved database configuration
- Improved HSM integration documentation
### 🚀 Features
- Add Utimaco General purpose HSM support
### 🐛 Bug Fixes
- Fixed HSM base code dangling pointer issue in `release` mode
- Fixed unwanted `ValueEnum` in `cosmian sym encrypt`
## [4.21.2] - 2025-01-21
### 📚 Documentation

232
Cargo.lock generated
View file

@ -8,7 +8,7 @@ version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5f7b0a21988c1bf877cf4759ef5ddaac04c1c9fe808c9142ecb78ba97d97a28a"
dependencies = [
"bitflags 2.6.0",
"bitflags 2.8.0",
"bytes",
"futures-core",
"futures-sink",
@ -47,7 +47,7 @@ dependencies = [
"actix-utils",
"ahash 0.8.11",
"base64 0.22.1",
"bitflags 2.6.0",
"bitflags 2.8.0",
"brotli",
"bytes",
"bytestring",
@ -462,9 +462,9 @@ checksum = "96d30a06541fbafbc7f82ed10c06164cfbd2c401138f6addd8404629c4b16711"
[[package]]
name = "asn1-rs"
version = "0.6.2"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5493c3bedbacf7fd7382c6346bbd66687d12bbaad3a89a2d2c303ee6cf20b048"
checksum = "607495ec7113b178fbba7a6166a27f99e774359ef4823adbefd756b5b81d7970"
dependencies = [
"asn1-rs-derive",
"asn1-rs-impl",
@ -472,15 +472,15 @@ dependencies = [
"nom",
"num-traits",
"rusticata-macros",
"thiserror 1.0.69",
"thiserror 2.0.11",
"time",
]
[[package]]
name = "asn1-rs-derive"
version = "0.5.1"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "965c2d33e53cb6b267e148a4cb0760bc01f4904c1cd4bb4002a085bb016d1490"
checksum = "3109e49b1e4909e9db6515a30c633684d68cdeaa252f215214cb4fa1a5bfee2c"
dependencies = [
"proc-macro2",
"quote",
@ -550,9 +550,9 @@ dependencies = [
[[package]]
name = "async-trait"
version = "0.1.81"
version = "0.1.86"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e0c28dcc82d7c8ead5cb13beb15405b57b8546e93215673ff8ca0349a028107"
checksum = "644dd749086bf3771a2fbc5f256fdb982d53f011c7d5d560304eafeecebce79d"
dependencies = [
"proc-macro2",
"quote",
@ -685,7 +685,7 @@ version = "0.69.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "271383c67ccabffb7381723dea0672a673f292304fcb45c01cc648c7a8d58088"
dependencies = [
"bitflags 2.6.0",
"bitflags 2.8.0",
"cexpr",
"clang-sys",
"itertools",
@ -716,9 +716,9 @@ checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a"
[[package]]
name = "bitflags"
version = "2.6.0"
version = "2.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b048fb63fd8b5923fc5aa7b340d8e156aec7ec02f0c78fa8a6ddc2613f6f71de"
checksum = "8f68f53c83ab957f72c32642f3868eec03eb974d1fb82e453128456482613d36"
dependencies = [
"serde",
]
@ -887,9 +887,9 @@ dependencies = [
[[package]]
name = "chrono"
version = "0.4.38"
version = "0.4.39"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a21f936df1771bf62b77f047b726c4625ff2e8aa607c01ec06e5a05bd8463401"
checksum = "7e36cc9d416881d2e24f9a963be5fb1cd90966419ac844274161d10488b3e825"
dependencies = [
"android-tzdata",
"iana-time-zone",
@ -953,7 +953,7 @@ dependencies = [
"pkcs1",
"serde_json",
"sha3",
"thiserror 2.0.3",
"thiserror 2.0.11",
"tokio",
"tracing",
"tracing-error",
@ -976,9 +976,9 @@ dependencies = [
[[package]]
name = "clap"
version = "4.5.21"
version = "4.5.27"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fb3b4b9e5a7c7514dfa52869339ee98b3156b0bfb4e8a77c4ff4babb64b1604f"
checksum = "769b0145982b4b48713e01ec42d61614425f27b7058bda7180a3a41f30104796"
dependencies = [
"clap_builder",
"clap_derive",
@ -986,9 +986,9 @@ dependencies = [
[[package]]
name = "clap_builder"
version = "4.5.21"
version = "4.5.27"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b17a95aa67cc7b5ebd32aa5370189aa0d79069ef1c64ce893bd30fb24bff20ec"
checksum = "1b26884eb4b57140e4d2d93652abfa49498b938b3c9179f9fc487b0acc3edad7"
dependencies = [
"anstyle",
"clap_lex",
@ -996,9 +996,9 @@ dependencies = [
[[package]]
name = "clap_derive"
version = "4.5.18"
version = "4.5.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4ac6a0c7b1a9e9a5186361f67dfa1b88213572f427fb9ab038efb2bd8c582dab"
checksum = "54b755194d6389280185988721fffba69495eed5ee9feeee9a599b53db80318c"
dependencies = [
"heck 0.5.0",
"proc-macro2",
@ -1008,9 +1008,9 @@ dependencies = [
[[package]]
name = "clap_lex"
version = "0.7.2"
version = "0.7.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1462739cb27611015575c0c11df5df7601141071f07518d56fcc1be504cbec97"
checksum = "f46ad14479a25103f283c0f10005961cf086d8dc42205bb44c46ac563475dca6"
[[package]]
name = "cloudproof"
@ -1359,18 +1359,18 @@ dependencies = [
name = "cosmian_kmip"
version = "4.21.2"
dependencies = [
"bitflags 2.6.0",
"bitflags 2.8.0",
"chrono",
"cosmian_logger",
"hex",
"leb128",
"num-bigint-dig",
"pyo3",
"rand 0.8.5",
"rand 0.9.0",
"serde",
"serde_json",
"strum",
"thiserror 2.0.3",
"thiserror 2.0.11",
"time",
"tracing",
"uuid",
@ -1385,12 +1385,28 @@ dependencies = [
"serde",
]
[[package]]
name = "cosmian_kms_base_hsm"
version = "4.21.2"
dependencies = [
"async-trait",
"cosmian_kms_interfaces",
"libloading",
"lru",
"pkcs11_sys",
"rand 0.9.0",
"thiserror 2.0.11",
"tracing",
"tracing-subscriber",
"zeroize",
]
[[package]]
name = "cosmian_kms_cli"
version = "4.21.2"
dependencies = [
"assert_cmd",
"base64 0.21.7",
"base64 0.22.1",
"clap",
"cloudproof",
"cosmian_config_utils",
@ -1412,7 +1428,7 @@ dependencies = [
"serde_json",
"strum",
"tempfile",
"thiserror 2.0.3",
"thiserror 2.0.11",
"tokio",
"tracing",
"url",
@ -1437,7 +1453,7 @@ dependencies = [
"reqwest",
"serde",
"serde_json",
"thiserror 2.0.3",
"thiserror 2.0.11",
"tracing",
"url",
"zeroize",
@ -1449,7 +1465,7 @@ version = "4.21.2"
dependencies = [
"aes-gcm-siv",
"argon2",
"base64 0.21.7",
"base64 0.22.1",
"cloudproof",
"cosmian_kmip",
"cosmian_logger",
@ -1460,7 +1476,7 @@ dependencies = [
"rust-ini",
"serde",
"serde_json",
"thiserror 2.0.3",
"thiserror 2.0.11",
"tracing",
"x509-parser",
"zeroize",
@ -1474,7 +1490,7 @@ dependencies = [
"cosmian_kmip",
"num-bigint-dig",
"serde_json",
"thiserror 2.0.3",
"thiserror 2.0.11",
"tracing",
"zeroize",
]
@ -1506,7 +1522,7 @@ dependencies = [
"actix-web",
"alcoholic_jwt",
"async-recursion",
"base64 0.21.7",
"base64 0.22.1",
"chrono",
"clap",
"cloudproof",
@ -1535,7 +1551,7 @@ dependencies = [
"serde_json",
"strum",
"tempfile",
"thiserror 2.0.3",
"thiserror 2.0.11",
"time",
"tokio",
"toml",
@ -1543,6 +1559,7 @@ dependencies = [
"tracing-opentelemetry",
"tracing-subscriber",
"url",
"utimaco_pkcs11_loader",
"uuid",
"x509-parser",
"zeroize",
@ -1570,7 +1587,7 @@ dependencies = [
"serde_json",
"sqlx",
"tempfile",
"thiserror 2.0.3",
"thiserror 2.0.11",
"tiny-keccak",
"tokio",
"tracing",
@ -1605,7 +1622,7 @@ dependencies = [
"rsa",
"serial_test",
"strum_macros 0.26.4",
"thiserror 2.0.3",
"thiserror 2.0.11",
"tracing",
"tracing-subscriber",
"zeroize",
@ -1818,9 +1835,9 @@ dependencies = [
[[package]]
name = "der-parser"
version = "9.0.0"
version = "10.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5cd0a5c643689626bec213c4d8bd4d96acc8ffdb4ad4bb6bc16abf27d5f4b553"
checksum = "07da5016415d5a3c4dd39b11ed26f915f52fc4e0dc197d87908bc916e51bc1a6"
dependencies = [
"asn1-rs",
"displaydoc",
@ -2270,16 +2287,13 @@ dependencies = [
[[package]]
name = "getrandom"
version = "0.3.0-rc.0"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8a78f88e84d239c7f2619ae8b091603c26208e1cb322571f5a29d6806f56ee5e"
checksum = "43a49c392881ce6d5c3b8cb70f98717b7c07aabbdff06687b9030dbfbe2725f8"
dependencies = [
"cfg-if",
"js-sys",
"libc",
"rustix",
"wasi 0.13.3+wasi-0.2.2",
"wasm-bindgen",
"windows-targets 0.52.6",
]
@ -2870,7 +2884,7 @@ name = "kms_test_server"
version = "4.21.2"
dependencies = [
"actix-server",
"base64 0.21.7",
"base64 0.22.1",
"cosmian_kms_client",
"cosmian_kms_crypto",
"cosmian_kms_server",
@ -2985,9 +2999,9 @@ dependencies = [
[[package]]
name = "log"
version = "0.4.22"
version = "0.4.25"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a7a70ba024b9dc04c27ea2f0c0548feb474ec5c54bba33a7f72f873a39d07b24"
checksum = "04cbf5b083de1c7e0222a7a51dbfdba1cbe1c6ab0b15e29fff3f6c077fd9cd9f"
[[package]]
name = "lru"
@ -3244,9 +3258,9 @@ dependencies = [
[[package]]
name = "oid-registry"
version = "0.7.0"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1c958dd45046245b9c3c2547369bb634eb461670b2e7e0de552905801a648d1d"
checksum = "264c56d1492c13e769662197fb6b94e0a52abe52d27efac374615799a4bf453d"
dependencies = [
"asn1-rs",
]
@ -3271,11 +3285,11 @@ checksum = "c08d65885ee38876c4f86fa503fb49d7b507c2b62552df7c70b2fce627e06381"
[[package]]
name = "openssl"
version = "0.10.68"
version = "0.10.70"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6174bc48f102d208783c2c84bf931bb75927a617866870de8a4ea85597f871f5"
checksum = "61cfb4e166a8bb8c9b55c500bc2308550148ece889be90f609377e58140f42c6"
dependencies = [
"bitflags 2.6.0",
"bitflags 2.8.0",
"cfg-if",
"foreign-types",
"libc",
@ -3303,9 +3317,9 @@ checksum = "ff011a302c396a5197692431fc1948019154afc178baf7d8e37367442a4601cf"
[[package]]
name = "openssl-sys"
version = "0.9.104"
version = "0.9.105"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "45abf306cbf99debc8195b66b7346498d7b10c210de50418b5ccd7ceba08c741"
checksum = "8b22d5b84be05a8d6947c7cb71f7c849aa0f112acd4bf51c2a7c1c988ac0a9dc"
dependencies = [
"cc",
"libc",
@ -3760,18 +3774,12 @@ dependencies = [
name = "proteccio_pkcs11_loader"
version = "4.21.2"
dependencies = [
"async-trait",
"cosmian_kms_base_hsm",
"cosmian_kms_interfaces",
"libloading",
"lru",
"pkcs11_sys",
"rand 0.9.0-beta.1",
"serial_test",
"thiserror 2.0.3",
"tracing",
"tracing-subscriber",
"uuid",
"zeroize",
]
[[package]]
@ -3882,13 +3890,13 @@ dependencies = [
[[package]]
name = "rand"
version = "0.9.0-beta.1"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c8478de76992f2825a1052cc2ae9d1401cdb62687761d4100ddd69a73dc3dc48"
checksum = "3779b94aeb87e8bd4e834cee3650289ee9e0d5677f976ecdb6d219e5f4f6cd94"
dependencies = [
"rand_chacha 0.9.0-beta.1",
"rand_core 0.9.0-beta.1",
"zerocopy 0.8.11",
"rand_chacha 0.9.0",
"rand_core 0.9.0",
"zerocopy 0.8.14",
]
[[package]]
@ -3903,12 +3911,12 @@ dependencies = [
[[package]]
name = "rand_chacha"
version = "0.9.0-beta.1"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f16da77124f4ee9fabd55ce6540866e9101431863b4876de58b68797f331adf2"
checksum = "d3022b5f1df60f26e1ffddd6c66e8aa15de382ae63b3a0c1bfc0e4d3e3f325cb"
dependencies = [
"ppv-lite86",
"rand_core 0.9.0-beta.1",
"rand_core 0.9.0",
]
[[package]]
@ -3922,12 +3930,12 @@ dependencies = [
[[package]]
name = "rand_core"
version = "0.9.0-beta.1"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a98fa0b8309344136abe6244130311e76997e546f76fae8054422a7539b43df7"
checksum = "b08f3c9802962f7e1b25113931d94f43ed9725bebc59db9d0c3e9a23b67e15ff"
dependencies = [
"getrandom 0.3.0-rc.0",
"zerocopy 0.8.11",
"getrandom 0.3.1",
"zerocopy 0.8.14",
]
[[package]]
@ -3977,7 +3985,7 @@ version = "0.5.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9b6dfecf2c74bce2466cabf93f6664d6998a69eb21e39f4207930065b27b771f"
dependencies = [
"bitflags 2.6.0",
"bitflags 2.8.0",
]
[[package]]
@ -4167,7 +4175,7 @@ version = "0.38.41"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d7f649912bc1495e167a6edee79151c84b1bad49748cb4f1f1167f459f6224f6"
dependencies = [
"bitflags 2.6.0",
"bitflags 2.8.0",
"errno",
"libc",
"linux-raw-sys",
@ -4306,7 +4314,7 @@ version = "2.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "897b2245f0b511c87893af39b033e5ca9cce68824c4d7e7630b5a1d339658d02"
dependencies = [
"bitflags 2.6.0",
"bitflags 2.8.0",
"core-foundation",
"core-foundation-sys",
"libc",
@ -4331,18 +4339,18 @@ checksum = "61697e0a1c7e512e84a621326239844a24d8207b4669b41bc18b32ea5cbf988b"
[[package]]
name = "serde"
version = "1.0.215"
version = "1.0.217"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6513c1ad0b11a9376da888e3e0baa0077f1aed55c17f50e7b2397136129fb88f"
checksum = "02fc4265df13d6fa1d00ecff087228cc0a2b5f3c0e87e258d8b94a156e984c70"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.215"
version = "1.0.217"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ad1e866f866923f252f05c889987993144fb74e722403468a4ebd70c3cd756c0"
checksum = "5a9bf7cf98d04a2b28aead066b7496853d4779c9cc183c440dbac457641e19a0"
dependencies = [
"proc-macro2",
"quote",
@ -4351,9 +4359,9 @@ dependencies = [
[[package]]
name = "serde_json"
version = "1.0.133"
version = "1.0.138"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c7fceb2473b9166b2294ef05efcb65a3db80803f0b03ef86a5fc88a2b85ee377"
checksum = "d434192e7da787e94a6ea7e9670b26a036d0ca41e0b7efb2676dd32bae872949"
dependencies = [
"indexmap 2.7.0",
"itoa",
@ -4658,7 +4666,7 @@ checksum = "64bb4714269afa44aef2755150a0fc19d756fb580a67db8885608cf02f47d06a"
dependencies = [
"atoi",
"base64 0.22.1",
"bitflags 2.6.0",
"bitflags 2.8.0",
"byteorder",
"bytes",
"crc",
@ -4700,7 +4708,7 @@ checksum = "6fa91a732d854c5d7726349bb4bb879bb9478993ceb764247660aee25f67c2f8"
dependencies = [
"atoi",
"base64 0.22.1",
"bitflags 2.6.0",
"bitflags 2.8.0",
"byteorder",
"crc",
"dotenvy",
@ -4887,12 +4895,13 @@ checksum = "61c41af27dd6d1e27b1b16b489db798443478cef1f06a660c96db617ba5de3b1"
[[package]]
name = "tempfile"
version = "3.14.0"
version = "3.16.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "28cce251fcbc87fac86a866eeb0d6c2d536fc16d06f184bb61aeae11aa4cee0c"
checksum = "38c246215d7d24f48ae091a2902398798e05d978b24315d6efbc00ede9a8bb91"
dependencies = [
"cfg-if",
"fastrand",
"getrandom 0.3.1",
"once_cell",
"rustix",
"windows-sys 0.59.0",
@ -4915,11 +4924,11 @@ dependencies = [
[[package]]
name = "thiserror"
version = "2.0.3"
version = "2.0.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c006c85c7651b3cf2ada4584faa36773bd07bac24acfb39f3c431b36d7e667aa"
checksum = "d452f284b73e6d76dd36758a0c8684b1d5be31f92b89d07fd5822175732206fc"
dependencies = [
"thiserror-impl 2.0.3",
"thiserror-impl 2.0.11",
]
[[package]]
@ -4935,9 +4944,9 @@ dependencies = [
[[package]]
name = "thiserror-impl"
version = "2.0.3"
version = "2.0.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f077553d607adc1caf65430528a576c757a71ed73944b66ebb58ef2bbd243568"
checksum = "26afc1baea8a989337eeb52b6e72a039780ce45c3edfcc9c5b9d112feeb173c2"
dependencies = [
"proc-macro2",
"quote",
@ -4956,9 +4965,9 @@ dependencies = [
[[package]]
name = "time"
version = "0.3.36"
version = "0.3.37"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5dfd88e563464686c916c7e46e623e520ddc6d79fa6641390f2e3fa86e83e885"
checksum = "35e7868883861bd0e56d9ac6efcaaca0d6d5d82a2a7ec8209ff492c07cf37b21"
dependencies = [
"deranged",
"itoa",
@ -4979,9 +4988,9 @@ checksum = "ef927ca75afb808a4d64dd374f00a2adf8d0fcff8e7b184af886c3c87ec4a3f3"
[[package]]
name = "time-macros"
version = "0.2.18"
version = "0.2.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f252a68540fde3a3877aeea552b832b40ab9a69e318efd078774a01ddee1ccf"
checksum = "2834e6017e3e5e4b9834939793b282bc03b37a3336245fa820e35e233e2a85de"
dependencies = [
"num-conv",
"time-core",
@ -5449,11 +5458,24 @@ version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be"
[[package]]
name = "utimaco_pkcs11_loader"
version = "4.21.2"
dependencies = [
"cosmian_kms_base_hsm",
"cosmian_kms_interfaces",
"libloading",
"pkcs11_sys",
"tracing",
"tracing-subscriber",
"uuid",
]
[[package]]
name = "uuid"
version = "1.11.0"
version = "1.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8c5f0a0af699448548ad1a2fbf920fb4bee257eae39953ba95cb84891a0446a"
checksum = "b3758f5e68192bb96cc8f9b7e2c2cfdabb435499a28499a42f8f984092adad4b"
dependencies = [
"getrandom 0.2.15",
]
@ -5887,7 +5909,7 @@ version = "0.33.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3268f3d866458b787f390cf61f4bbb563b922d091359f9608842999eaee3943c"
dependencies = [
"bitflags 2.6.0",
"bitflags 2.8.0",
]
[[package]]
@ -5919,9 +5941,9 @@ dependencies = [
[[package]]
name = "x509-parser"
version = "0.16.0"
version = "0.17.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fcbc162f30700d6f3f82a24bf7cc62ffe7caea42c0b2cba8bf7f3ae50cf51f69"
checksum = "4569f339c0c402346d4a75a9e39cf8dad310e287eef1ff56d4c68e5067f53460"
dependencies = [
"asn1-rs",
"data-encoding",
@ -5931,7 +5953,7 @@ dependencies = [
"oid-registry",
"ring",
"rusticata-macros",
"thiserror 1.0.69",
"thiserror 2.0.11",
"time",
]
@ -5971,11 +5993,11 @@ dependencies = [
[[package]]
name = "zerocopy"
version = "0.8.11"
version = "0.8.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cce3b5629d87654b53a49002acc2ce64aa5aa7255f5c718374a37ac7fd98c218"
checksum = "a367f292d93d4eab890745e75a778da40909cab4d6ff8173693812f79c4a2468"
dependencies = [
"zerocopy-derive 0.8.11",
"zerocopy-derive 0.8.14",
]
[[package]]
@ -5991,9 +6013,9 @@ dependencies = [
[[package]]
name = "zerocopy-derive"
version = "0.8.11"
version = "0.8.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "74a82c26c3986af2623ec9eb890ff4aa19c006e30a1133dc9bd1830ec1612e20"
checksum = "d3931cb58c62c13adec22e38686b559c86a30565e16ad6e8510a337cedc611e1"
dependencies = [
"proc-macro2",
"quote",

36
Cargo.toml
View file

@ -5,12 +5,14 @@ members = [
"crate/cli",
"crate/client",
"crate/crypto",
"crate/hsm/proteccio",
"crate/hsm/utimaco",
"crate/hsm/base_hsm",
"crate/interfaces",
"crate/kmip",
"crate/pyo3",
"crate/pkcs11/module",
"crate/pkcs11/provider",
"crate/pkcs11/proteccio",
"crate/pkcs11/sys",
"crate/server",
"crate/server_database",
@ -28,7 +30,6 @@ rust-version = "1.71.0"
authors = [
"Bruno Grieder <bruno.grieder@cosmian.com>",
"Emmanuel Coste <emmanuel.coste@cosmian.com>",
"Hugo Rosenkranz-Costa <hugo.rosenkranz@cosmian.com>",
]
license = "BUSL-1.1" # "Business Source License 1.1"
license-file = "LICENSE"
@ -57,11 +58,11 @@ opt-level = 0
actix-rt = "2.10"
actix-server = { version = "2.5", default-features = false }
actix-web = { version = "4.9", default-features = false }
async-trait = "0.1"
base64 = "0.21"
bitflags = "2.6"
chrono = "0.4"
clap = { version = "4.5", default-features = false }
async-trait = "0.1.86"
base64 = "0.22.1"
bitflags = "2.8.0"
chrono = "0.4.39"
clap = { version = "4.5.27", default-features = false }
cloudproof = "3.0"
cloudproof_findex = { version = "5.0", features = ["findex-redis"] }
cosmian_config_utils = { git = "https://www.github.com/Cosmian/http_client_server", branch = "develop" }
@ -71,28 +72,29 @@ der = { version = "0.7", default-features = false }
hex = { version = "0.4", default-features = false }
lazy_static = "1.5"
leb128 = "0.2"
log = { version = "0.4", default-features = false }
libloading = "0.8.6"
log = { version = "0.4.25", default-features = false }
lru = "0.12.5"
num_cpus = "1.16"
num-format = "0.4"
num-bigint-dig = { version = "0.8", default-features = false }
openssl = { version = "0.10", default-features = false }
openssl = { version = "0.10.70", default-features = false }
pem = "3.0"
pyo3 = { version = "0.20", default-features = false }
rand = "0.8"
rand = "0.9"
reqwest = { version = "0.11", default-features = false }
serde = "1.0"
serde_json = "1.0"
serde = "1.0.217"
serde_json = "1.0.138"
sha3 = { version = "0.10", default-features = false }
strum = { version = "0.25", default-features = false }
thiserror = "2.0"
time = "0.3"
tempfile = "3"
thiserror = "2.0.11"
time = "0.3.37"
tempfile = "3.16.0"
tokio = { version = "1.42", default-features = false }
tracing-subscriber = { version = "0.3", default-features = false }
tracing = "0.1"
url = "2.5"
uuid = "1.11"
uuid = "1.12.1"
x509-cert = { version = "0.2", default-features = false }
x509-parser = "0.16"
x509-parser = "0.17.0"
zeroize = { version = "1.8", default-features = false }

9
README.md
View file

@ -10,19 +10,20 @@ written in [**Rust**](https://www.rust-lang.org/) that presents some unique feat
- the ability to confidentially run in a public cloud — or any zero-trust environment — using
Cosmian VM. See our cloud-ready confidential KMS on the
[Azure, GCP, and AWS marketplaces](https://cosmian.com/marketplaces/) and our [deployment guide](./documentation/docs/marketplace_guide.md)
[Azure, GCP, and AWS marketplaces](https://cosmian.com/marketplaces/) and
our [deployment guide](documentation/docs/installation/marketplace_guide.md)
- support of state-of-the-art authentication mechanisms (see [authentication](./documentation/docs/authentication.md))
- out-of-the-box support of
[Google Workspace Client Side Encryption (CSE)](./documentation/docs/google_cse/index.md)
- out-of-the-box support
of [Microsoft Double Key Encryption (DKE)](./documentation/docs/ms_dke/index.md)
- support for the [Proteccio HSM](./documentation/docs/hsm.md) with KMS keys wrapped by the HSM
- support for the [Proteccio HSM](./documentation/docs/hsms/index.md) with KMS keys wrapped by the HSM
- [Veracrypt](./documentation/docs/pkcs11/veracrypt.md)
and [LUKS](./documentation/docs/pkcs11/luks.md) disk encryption support
- [FIPS 140-3](./documentation/docs/fips.md) mode gated behind the feature `fips`
- a [JSON KMIP 2.1](./documentation/docs/kmip_2_1/index.md) compliant interface
- a full-featured client [command line and graphical interface](https://docs.cosmian.com/cosmian_cli/)
- a [high-availability mode](./documentation/docs/high_availability_mode.md) with simple horizontal scaling
- a [high-availability mode](documentation/docs/installation/high_availability_mode.md) with simple horizontal scaling
- a support of Python, Javascript, Dart, Rust, C/C++, and Java clients (see the `cloudproof` libraries
on [Cosmian Github](https://github.com/Cosmian))
- integrated with [OpenTelemetry](https://opentelemetry.io/)
@ -268,7 +269,7 @@ Otherwise, the parameters are set following this order:
## Use the KMS inside a Cosmian VM on SEV/TDX
See the [Marketplace guide](./documentation/docs/marketplace_guide.md) for more details about Cosmian VM.
See the [Marketplace guide](documentation/docs/installation/marketplace_guide.md) for more details about Cosmian VM.
## Releases

8
crate/cli/Cargo.toml
View file

@ -19,7 +19,11 @@ test = false
doctest = false
[features]
fips = ["cosmian_kms_client/fips", "cosmian_kms_crypto/fips", "kms_test_server/fips"]
fips = [
"cosmian_kms_client/fips",
"cosmian_kms_crypto/fips",
"kms_test_server/fips",
]
[dependencies]
base64 = { workspace = true }
@ -46,7 +50,7 @@ leb128 = { workspace = true }
num-format = { workspace = true }
pem = { workspace = true }
reqwest = { workspace = true }
serde = { workspace = true }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
strum = { workspace = true, features = ["std", "derive", "strum_macros"] }
thiserror = { workspace = true }

20
crate/cli/src/actions/symmetric/decrypt.rs
View file

@ -76,6 +76,11 @@ pub struct DecryptAction {
#[clap(long = "key-id", short = 'k', group = "key-tags")]
key_id: Option<String>,
/// Tag to use to retrieve the key when no key id is specified.
/// To specify multiple tags, use the option multiple times.
#[clap(long = "tag", short = 't', value_name = "TAG", group = "key-tags")]
tags: Option<Vec<String>>,
/// The data encryption algorithm.
/// If not specified, aes-gcm is used.
///
@ -84,7 +89,8 @@ pub struct DecryptAction {
#[clap(
long = "data-encryption-algorithm",
short = 'd',
default_value = "aes-gcm"
default_value = "aes-gcm",
verbatim_doc_comment
)]
data_encryption_algorithm: DataEncryptionAlgorithm,
@ -102,17 +108,12 @@ pub struct DecryptAction {
#[clap(long = "key-encryption-algorithm", short = 'e', verbatim_doc_comment)]
key_encryption_algorithm: Option<KeyEncryptionAlgorithm>,
/// Tag to use to retrieve the key when no key id is specified.
/// To specify multiple tags, use the option multiple times.
#[clap(long = "tag", short = 't', value_name = "TAG", group = "key-tags")]
tags: Option<Vec<String>>,
/// The encrypted output file path
#[clap(required = false, long, short = 'o')]
#[clap(long, short = 'o')]
output_file: Option<PathBuf>,
/// Optional authentication data that was supplied during encryption as a hex string.
#[clap(required = false, long, short)]
#[clap(long, short = 'a')]
authentication_data: Option<String>,
}
@ -217,6 +218,9 @@ impl DecryptAction {
}
a => cli_bail!("Unsupported cryptographic algorithm: {:?}", a),
};
if nonce_size + tag_size > ciphertext.len() {
cli_bail!("The ciphertext is too short to contain the nonce/tweak and the tag")
}
let nonce = ciphertext.drain(..nonce_size).collect::<Vec<_>>();
let tag = ciphertext
.drain(ciphertext.len() - tag_size..)

2
crate/cli/src/actions/symmetric/encrypt.rs
View file

@ -66,7 +66,7 @@ pub struct EncryptAction {
key_id: Option<String>,
/// The data encryption algorithm.
/// If not specified, aes-gcm is used.
/// If not specified, `aes-gcm` is used.
///
/// If no key encryption algorithm is specified, the data will be sent to the server
/// and will be encrypted server side.

17
crate/cli/src/actions/symmetric/mod.rs
View file

@ -3,7 +3,7 @@ use cosmian_kms_client::{
kmip_2_1::kmip_types::{BlockCipherMode, CryptographicAlgorithm, CryptographicParameters},
KmsClient,
};
use strum::{Display, EnumIter};
use strum::EnumIter;
pub use self::{decrypt::DecryptAction, encrypt::EncryptAction, keys::KeysCommands};
use crate::error::result::CliResult;
@ -42,18 +42,15 @@ impl SymmetricCommands {
}
}
#[derive(ValueEnum, Debug, Clone, Copy, Default, EnumIter, PartialEq, Eq, Display)]
#[derive(ValueEnum, Debug, Clone, Copy, Default, EnumIter, PartialEq, Eq, strum::Display)]
#[strum(serialize_all = "kebab-case")]
pub enum DataEncryptionAlgorithm {
#[cfg(not(feature = "fips"))]
#[value(name = "Chacha20Poly1305")]
Chacha20Poly1305,
#[default]
#[value(name = "AesGcm")]
AesGcm,
#[value(name = "AesXts")]
AesXts,
#[cfg(not(feature = "fips"))]
#[value(name = "AesGcmSiv")]
AesGcmSiv,
}
@ -85,19 +82,15 @@ impl From<DataEncryptionAlgorithm> for CryptographicParameters {
}
}
#[derive(ValueEnum, Debug, Clone, Copy, EnumIter, Display)]
#[derive(ValueEnum, Debug, Clone, Copy, EnumIter, strum::Display)]
#[strum(serialize_all = "kebab-case")]
pub enum KeyEncryptionAlgorithm {
#[cfg(not(feature = "fips"))]
#[value(name = "Chacha20Poly1305")]
Chacha20Poly1305,
#[value(name = "AesGcm")]
AesGcm,
#[value(name = "AesXts")]
AesXts,
#[cfg(not(feature = "fips"))]
#[value(name = "AesGcmSiv")]
AesGcmSiv,
#[value(name = "RFC5649")]
RFC5649,
}

17
crate/pkcs11/proteccio/Cargo.toml → crate/hsm/base_hsm/Cargo.toml
View file

@ -1,12 +1,12 @@
[package]
name = "proteccio_pkcs11_loader"
name = "cosmian_kms_base_hsm"
version.workspace = true
authors.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
rust-version.workspace = true
description = "Proteccio HSM PKCS#11 loader"
description = "Base PKCS#11 HSM integration implementation"
[lib]
doctest = false
@ -14,21 +14,14 @@ doctest = false
[dependencies]
async-trait = { workspace = true }
cosmian_kms_interfaces = { path = "../../interfaces" }
libloading = "0.8.6"
libloading = { workspace = true }
lru = { workspace = true }
pkcs11_sys = { path = "../sys" }
rand = "0.9.0-beta.1"
pkcs11_sys = { path = "../../pkcs11/sys" }
rand = { workspace = true }
thiserror = { workspace = true }
tracing = { workspace = true }
tracing-subscriber = { workspace = true }
uuid = { version = "1.11.0", features = ["v4"] }
zeroize = { workspace = true }
[dev-dependencies]
serial_test = { version = "3.2.0", default-features = false }
tracing-subscriber = { workspace = true, features = ["env-filter"] }
uuid = { workspace = true }
[features]
# Enable this feature to run Proteccio tests (they require a Proteccio HSM)
proteccio = []

21
crate/hsm/base_hsm/README.md Normal file
View file

@ -0,0 +1,21 @@
# Base HSM Implementation
This crate contains the implementation of a PKCS#11 client for a Hardware Security Modules (HSMs).
It provides a set of traits that define the operations that an HSM must support,
as well as a set of data structures that represent the keys and metadata that an HSM can manage.
## Implemented Operations
- Key Generation: Create symmetric (AES) and asymmetric (RSA) keys
- Key Pair Generation: Create public/private key pairs
- Key Export: Export HSM objects
- Key Deletion: Remove keys from the HSM
- Key Search: Find keys based on object type filters
- Encryption/Decryption: Perform cryptographic operations
- Key Information: Retrieve key types and metadata
## Supported Algorithms
- AES: 128-bit and 256-bit keys
- RSA: 1024-bit, 2048-bit, 3072-bit, and 4096-bit keys

135
crate/hsm/base_hsm/src/base_hsm.rs Normal file
View file

@ -0,0 +1,135 @@
use std::{
collections::HashMap,
ffi::CStr,
fmt,
fmt::{Display, Formatter},
sync::{Arc, Mutex},
};
use pkcs11_sys::{CKR_OK, CK_INFO};
use crate::{hsm_lib::HsmLib, HError, HResult, SlotManager};
struct SlotState {
password: Option<String>,
slot: Option<Arc<SlotManager>>,
}
pub struct BaseHsm {
hsm_lib: Arc<HsmLib>,
slots: Mutex<HashMap<usize, SlotState>>,
}
impl BaseHsm {
pub fn instantiate<P>(path: P, passwords: HashMap<usize, Option<String>>) -> HResult<Self>
where
P: AsRef<std::ffi::OsStr>,
{
let hsm_lib = Arc::new(HsmLib::instantiate(path)?);
let mut slots = HashMap::with_capacity(passwords.len());
for (k, v) in passwords.iter() {
slots.insert(
*k,
SlotState {
password: v.clone(),
slot: None,
},
);
}
Ok(BaseHsm {
hsm_lib,
slots: Mutex::new(slots),
})
}
/// Get a slot
/// If a slot has already been opened, returns the opened slot.
/// To close a slot before re-opening it with another password, call `close_slot()` first
pub fn get_slot(&self, slot_id: usize) -> HResult<Arc<SlotManager>> {
let mut slots = self.slots.lock().expect("failed to lock slots");
// check if we are supposed to use that slot
if let Some(slot_state) = slots.get_mut(&slot_id) {
if let Some(s) = &slot_state.slot {
Ok(s.clone())
} else {
// instantiate a new slot
let manager = Arc::new(SlotManager::instantiate(
self.hsm_lib.clone(),
slot_id,
slot_state.password.clone(),
)?);
slot_state.slot = Some(manager.clone());
Ok(manager)
}
} else {
Err(HError::Default(format!("slot {slot_id} is not accessible")))
}
}
pub fn close_slot(&self, slot_id: usize) -> HResult<()> {
let mut slots = self.slots.lock().expect("failed to lock slots");
slots.remove(&slot_id);
Ok(())
}
pub fn get_info(&self) -> HResult<Info> {
unsafe {
let mut info = CK_INFO::default();
let rv =
self.hsm_lib.C_GetInfo.ok_or_else(|| {
HError::Default("C_GetInfo not available on library".to_string())
})?(&mut info);
if rv != CKR_OK {
return Err(HError::Default("Failed getting HSM info".to_string()));
}
Ok(info.into())
}
}
}
pub struct Info {
pub cryptokiVersion: (u8, u8),
pub manufacturerID: String,
pub flags: u64,
pub libraryDescription: String,
pub libraryVersion: (u8, u8),
}
impl From<CK_INFO> for Info {
fn from(info: CK_INFO) -> Self {
#[cfg(target_os = "windows")]
let flags = u64::from(info.flags);
#[cfg(not(target_os = "windows"))]
let flags = info.flags;
Info {
cryptokiVersion: (info.cryptokiVersion.major, info.cryptokiVersion.minor),
manufacturerID: CStr::from_bytes_until_nul(&info.manufacturerID)
.unwrap_or_default()
.to_string_lossy()
.to_string(),
flags,
libraryDescription: CStr::from_bytes_until_nul(&info.libraryDescription)
.unwrap_or_default()
.to_string_lossy()
.to_string(),
libraryVersion: (info.libraryVersion.major, info.libraryVersion.minor),
}
}
}
impl Display for Info {
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
write!(
f,
"Cryptoki Version: {}.{}\nManufacturer ID: {}\nFlags: {}\nLibrary Description: \
{}\nLibrary Version: {}.{}",
self.cryptokiVersion.0,
self.cryptokiVersion.1,
self.manufacturerID,
self.flags,
self.libraryDescription,
self.libraryVersion.0,
self.libraryVersion.1
)
}
}

12
crate/pkcs11/proteccio/src/error.rs → crate/hsm/base_hsm/src/error.rs
View file

@ -3,10 +3,10 @@
use cosmian_kms_interfaces::InterfaceError;
use thiserror::Error;
pub type PResult<T> = Result<T, PError>;
pub type HResult<T> = Result<T, HError>;
#[derive(Error, Debug)]
pub enum PError {
pub enum HError {
#[error("{0}")]
Default(String),
@ -23,14 +23,14 @@ pub enum PError {
TryFromIntError(#[from] std::num::TryFromIntError),
}
impl From<InterfaceError> for PError {
impl From<InterfaceError> for HError {
fn from(e: InterfaceError) -> Self {
PError::Hsm(e.to_string())
HError::Hsm(e.to_string())
}
}
impl From<PError> for InterfaceError {
fn from(e: PError) -> Self {
impl From<HError> for InterfaceError {
fn from(e: HError) -> Self {
InterfaceError::Default(e.to_string())
}
}

166
crate/hsm/base_hsm/src/hsm_lib.rs Normal file
View file

@ -0,0 +1,166 @@
use std::ptr;
use libloading::Library;
use pkcs11_sys::*;
use crate::{HError, HResult};
/// A struct representing a Hardware Security Module (HSM) library interface using PKCS#11.
///
/// This struct provides a safe wrapper around the PKCS#11 library functions, managing
/// the dynamic loading of the HSM library and providing access to cryptographic operations.
///
/// # Fields
///
/// All fields are PKCS#11 function pointers that correspond to various cryptographic
/// and key management operations. The fields are marked as `pub(crate)` to allow
/// access within the crate while maintaining encapsulation.
///
/// # Examples
///
/// ```no_run
/// use hsm_common::HsmLib;
///
/// let hsm = HsmLib::instantiate("/path/to/hsm/library.so").expect("Failed to load HSM library");
/// ```
///
/// # Safety
///
/// This struct handles unsafe FFI calls to the PKCS#11 library internally. The public
/// interface is designed to be safe to use, but care must be taken when using the
/// raw function pointers directly.
///
/// The library automatically handles initialization and cleanup through the `Drop` trait,
/// ensuring proper finalization of the HSM when the struct is dropped.
///
/// # Methods
///
/// - `instantiate<P>`: Creates a new instance of the HSM library
/// - `initialize`: Initializes the HSM with OS locking capabilities
/// - `finalize`: Properly closes the HSM connection
///
/// # Error Handling
///
/// Operations return `PResult<T>`, which is a custom result type for handling
/// HSM-specific errors. Failed operations typically return `PError` variants
/// with descriptive error messages.
#[allow(dead_code)]
pub struct HsmLib {
_library: Library,
pub(crate) C_Initialize: CK_C_Initialize,
pub(crate) C_Finalize: CK_C_Finalize,
pub(crate) C_OpenSession: CK_C_OpenSession,
pub(crate) C_CloseSession: CK_C_CloseSession,
pub(crate) C_DestroyObject: CK_C_DestroyObject,
pub(crate) C_Decrypt: CK_C_Decrypt,
pub(crate) C_DecryptInit: CK_C_DecryptInit,
pub(crate) C_DecryptUpdate: CK_C_DecryptUpdate,
pub(crate) C_DecryptFinal: CK_C_DecryptFinal,
pub(crate) C_Encrypt: CK_C_Encrypt,
pub(crate) C_EncryptInit: CK_C_EncryptInit,
pub(crate) C_EncryptUpdate: CK_C_EncryptUpdate,
pub(crate) C_EncryptFinal: CK_C_EncryptFinal,
pub(crate) C_FindObjectsInit: CK_C_FindObjectsInit,
pub(crate) C_FindObjects: CK_C_FindObjects,
pub(crate) C_FindObjectsFinal: CK_C_FindObjectsFinal,
pub(crate) C_GenerateKey: CK_C_GenerateKey,
pub(crate) C_GenerateKeyPair: CK_C_GenerateKeyPair,
pub(crate) C_GenerateRandom: CK_C_GenerateRandom,
pub(crate) C_GetAttributeValue: CK_C_GetAttributeValue,
pub(crate) C_GetInfo: CK_C_GetInfo,
pub(crate) C_Login: CK_C_Login,
pub(crate) C_Logout: CK_C_Logout,
pub(crate) C_WrapKey: CK_C_WrapKey,
pub(crate) C_UnwrapKey: CK_C_UnwrapKey,
}
impl HsmLib {
pub(crate) fn instantiate<P>(path: P) -> HResult<Self>
where
P: AsRef<std::ffi::OsStr>,
{
unsafe {
let library = Library::new(path)?;
let hsm_lib = HsmLib {
C_Initialize: Some(*library.get(b"C_Initialize")?),
C_Finalize: Some(*library.get(b"C_Finalize")?),
C_OpenSession: Some(*library.get(b"C_OpenSession")?),
C_CloseSession: Some(*library.get(b"C_CloseSession")?),
C_Encrypt: Some(*library.get(b"C_Encrypt")?),
C_EncryptInit: Some(*library.get(b"C_EncryptInit")?),
C_EncryptUpdate: Some(*library.get(b"C_EncryptUpdate")?),
C_EncryptFinal: Some(*library.get(b"C_EncryptFinal")?),
C_Decrypt: Some(*library.get(b"C_Decrypt")?),
C_DecryptInit: Some(*library.get(b"C_DecryptInit")?),
C_DecryptUpdate: Some(*library.get(b"C_DecryptUpdate")?),
C_DecryptFinal: Some(*library.get(b"C_DecryptFinal")?),
C_DestroyObject: Some(*library.get(b"C_DestroyObject")?),
C_FindObjectsInit: Some(*library.get(b"C_FindObjectsInit")?),
C_FindObjects: Some(*library.get(b"C_FindObjects")?),
C_FindObjectsFinal: Some(*library.get(b"C_FindObjectsFinal")?),
C_GenerateKey: Some(*library.get(b"C_GenerateKey")?),
C_GenerateKeyPair: Some(*library.get(b"C_GenerateKeyPair")?),
C_GenerateRandom: Some(*library.get(b"C_GenerateRandom")?),
C_GetAttributeValue: Some(*library.get(b"C_GetAttributeValue")?),
C_GetInfo: Some(*library.get(b"C_GetInfo")?),
C_Login: Some(*library.get(b"C_Login")?),
C_Logout: Some(*library.get(b"C_Logout")?),
C_WrapKey: Some(*library.get(b"C_WrapKey")?),
C_UnwrapKey: Some(*library.get(b"C_UnwrapKey")?),
// we need to keep the library alive
_library: library,
};
Self::initialize(&hsm_lib)?;
Ok(hsm_lib)
}
}
fn initialize(hsm_lib: &HsmLib) -> HResult<()> {
let pInitArgs = CK_C_INITIALIZE_ARGS {
CreateMutex: None,
DestroyMutex: None,
LockMutex: None,
UnlockMutex: None,
flags: CKF_OS_LOCKING_OK,
pReserved: ptr::null_mut(),
};
unsafe {
// let rv = self.hsm.C_Initialize.deref()(&pInitArgs);
let rv = hsm_lib.C_Initialize.ok_or_else(|| {
HError::Default("C_Initialize not available on library".to_string())
})?(&pInitArgs as *const CK_C_INITIALIZE_ARGS as CK_VOID_PTR);
if rv != CKR_OK {
return Err(HError::Default("Failed initializing the HSM".to_string()));
}
Ok(())
}
}
fn finalize(&self) -> HResult<()> {
unsafe {
let rv = self.C_Finalize.ok_or_else(|| {
HError::Default("C_Finalize not available on library".to_string())
})?(ptr::null_mut());
if rv != CKR_OK {
return Err(HError::Default("Failed to finalize the HSM".to_string()));
}
Ok(())
}
}
}
impl Drop for HsmLib {
fn drop(&mut self) {
let _ = self.finalize();
}
}

45
crate/pkcs11/proteccio/src/kms_hsm.rs → crate/hsm/base_hsm/src/kms_hsm.rs
View file

@ -1,3 +1,36 @@
//! Implementation of the Hardware Security Module (HSM) trait for BaseHsm
//!
//! This implementation provides cryptographic operations using a Hardware Security Module,
//! supporting various key management and cryptographic operations.
//!
//! # Implemented Operations
//!
//! - Key Generation: Create symmetric (AES) and asymmetric (RSA) keys
//! - Key Pair Generation: Create public/private key pairs
//! - Key Export: Export HSM objects
//! - Key Deletion: Remove keys from the HSM
//! - Key Search: Find keys based on object type filters
//! - Encryption/Decryption: Perform cryptographic operations
//! - Key Information: Retrieve key types and metadata
//!
//! # Supported Algorithms
//!
//! - AES: 128-bit and 256-bit keys
//! - RSA: 1024-bit, 2048-bit, 3072-bit, and 4096-bit keys
//!
//! # Error Handling
//!
//! All operations return `InterfaceResult<T>` which may contain:
//! - Errors for duplicate key IDs
//! - Invalid key sizes
//! - Object not found errors
//! - General HSM operation failures
//!
//! # Security Features
//!
//! - Support for sensitive key material handling
//! - Secure session management
//! - Zero-copy cleanup for sensitive data using `Zeroizing`
use async_trait::async_trait;
use cosmian_kms_interfaces::{
CryptoAlgorithm, EncryptedContent, HsmKeyAlgorithm, HsmKeypairAlgorithm, HsmObject,
@ -5,10 +38,10 @@ use cosmian_kms_interfaces::{
};
use zeroize::Zeroizing;
use crate::{AesKeySize, Proteccio, RsaKeySize};
use crate::{AesKeySize, BaseHsm, RsaKeySize};
#[async_trait]
impl HSM for Proteccio {
impl HSM for BaseHsm {
async fn create_key(
&self,
slot_id: usize,
@ -39,9 +72,7 @@ impl HSM for Proteccio {
};
let _ = session.generate_aes_key(id, key_size, sensitive)?;
Ok(())
} // _ => Err(PluginError::Default(
// "Only AES or RSA keys can be created on the Proteccio HSM".to_string(),
// )),
}
}
}
@ -85,9 +116,7 @@ impl HSM for Proteccio {
HsmKeypairAlgorithm::RSA => {
session.generate_rsa_key_pair(sk_id, pk_id, key_length_in_bits, sensitive)?;
Ok(())
} // _ => Err(PluginError::Default(
// "Only AES or RSA keys can be created on the Proteccio HSM".to_string(),
// )),
}
}
}

84
crate/hsm/base_hsm/src/lib.rs Normal file
View file

@ -0,0 +1,84 @@
//! Copyright 2024 Cosmian Tech SAS
#![allow(non_snake_case)]
#![allow(clippy::missing_safety_doc)]
extern crate core;
mod error;
pub use base_hsm::BaseHsm;
pub use error::{HError, HResult};
pub use session::{AesKeySize, HsmEncryptionAlgorithm, RsaKeySize, Session};
pub use slots::{ObjectHandlesCache, SlotManager};
mod base_hsm;
mod hsm_lib;
mod session;
mod kms_hsm;
mod slots;
pub mod test_helpers;
// AES key template
// If sensitive is true, the key is not exportable
// Proteccio does not allow setting the ID attribute for secret keys so we use the LABEL
// so we do the same with other HSMs
#[macro_export]
macro_rules! aes_key_template {
($id:expr, $size:expr, $sensitive:expr) => {
[
CK_ATTRIBUTE {
type_: CKA_CLASS,
pValue: &CKO_SECRET_KEY as *const _ as CK_VOID_PTR,
ulValueLen: std::mem::size_of::<CK_ULONG>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_KEY_TYPE,
pValue: &CKK_AES as *const _ as CK_VOID_PTR,
ulValueLen: std::mem::size_of::<CK_ULONG>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_VALUE_LEN,
pValue: &$size as *const _ as CK_VOID_PTR,
ulValueLen: std::mem::size_of::<CK_ULONG>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_TOKEN,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: std::mem::size_of::<CK_BBOOL>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_ENCRYPT,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: std::mem::size_of::<CK_BBOOL>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_DECRYPT,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: std::mem::size_of::<CK_BBOOL>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_LABEL,
pValue: $id.as_ptr() as CK_VOID_PTR,
ulValueLen: $id.len() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_PRIVATE,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: std::mem::size_of::<CK_BBOOL>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_SENSITIVE,
pValue: &$sensitive as *const _ as CK_VOID_PTR,
ulValueLen: std::mem::size_of::<CK_BBOOL>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_EXTRACTABLE,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: std::mem::size_of::<CK_BBOOL>() as CK_ULONG,
},
]
};
}

65
crate/hsm/base_hsm/src/session/aes.rs Normal file
View file

@ -0,0 +1,65 @@
use std::ptr;
use pkcs11_sys::{
CKA_CLASS, CKA_DECRYPT, CKA_ENCRYPT, CKA_EXTRACTABLE, CKA_KEY_TYPE, CKA_LABEL, CKA_PRIVATE,
CKA_SENSITIVE, CKA_TOKEN, CKA_VALUE_LEN, CKK_AES, CKM_AES_KEY_GEN, CKO_SECRET_KEY, CKR_OK,
CK_ATTRIBUTE, CK_ATTRIBUTE_PTR, CK_BBOOL, CK_FALSE, CK_MECHANISM, CK_MECHANISM_PTR,
CK_OBJECT_HANDLE, CK_TRUE, CK_ULONG, CK_VOID_PTR,
};
use crate::{aes_key_template, session::Session, HError, HResult};
pub enum AesKeySize {
Aes128,
Aes256,
}
impl Session {
/// Generate an AES key
///
/// If exportable is set to `false`, the `sensitive` flag is set to true,
/// and the key will not be exportable.
pub fn generate_aes_key(
&self,
id: &[u8],
size: AesKeySize,
sensitive: bool,
) -> HResult<CK_OBJECT_HANDLE> {
unsafe {
let ck_fn = self.hsm().C_GenerateKey.ok_or_else(|| {
HError::Default("C_GenerateKey not available on library".to_string())
})?;
let size = match size {
AesKeySize::Aes128 => 16,
AesKeySize::Aes256 => 32,
} as CK_ULONG;
let mut mechanism = CK_MECHANISM {
mechanism: CKM_AES_KEY_GEN,
pParameter: ptr::null_mut(),
ulParameterLen: 0,
};
let is_sensitive = if !sensitive { CK_FALSE } else { CK_TRUE };
let mut template = aes_key_template!(id, size, is_sensitive);
let pMechanism: CK_MECHANISM_PTR = &mut mechanism;
let pMutTemplate: CK_ATTRIBUTE_PTR = template.as_mut_ptr();
let mut aes_key_handle = CK_OBJECT_HANDLE::default();
#[cfg(target_os = "windows")]
let len = u32::try_from(template.len())?;
#[cfg(not(target_os = "windows"))]
let len = u64::try_from(template.len())?;
let rv = ck_fn(
self.session_handle(),
pMechanism,
pMutTemplate,
len,
&mut aes_key_handle,
);
if rv != CKR_OK {
return Err(HError::Default(format!("Failed generating key: {rv}")));
}
self.object_handles_cache()
.insert(id.to_vec(), aes_key_handle);
Ok(aes_key_handle)
}
}
}

5
crate/hsm/base_hsm/src/session/mod.rs Normal file
View file

@ -0,0 +1,5 @@
mod aes;
mod rsa;
mod session_impl;
pub use session_impl::{AesKeySize, HsmEncryptionAlgorithm, RsaKeySize, Session};

122
crate/pkcs11/proteccio/src/session/rsa.rs → crate/hsm/base_hsm/src/session/rsa.rs
View file

@ -9,7 +9,7 @@ use pkcs11_sys::{
CK_RSA_PKCS_OAEP_PARAMS, CK_TRUE, CK_ULONG, CK_VOID_PTR,
};
use crate::{session::Session, PError, PResult};
use crate::{session::Session, HError, HResult};
pub enum RsaKeySize {
Rsa1024,
@ -38,7 +38,7 @@ impl Session {
pk_id: &[u8],
key_size: RsaKeySize,
sensitive: bool,
) -> PResult<(CK_OBJECT_HANDLE, CK_OBJECT_HANDLE)> {
) -> HResult<(CK_OBJECT_HANDLE, CK_OBJECT_HANDLE)> {
let key_size = match key_size {
RsaKeySize::Rsa1024 => 1024,
RsaKeySize::Rsa2048 => 2048,
@ -150,7 +150,7 @@ impl Session {
let pMechanism: CK_MECHANISM_PTR = &mut mechanism;
let rv = self.hsm().C_GenerateKeyPair.ok_or_else(|| {
PError::Default("C_GenerateKeyPair not available on library".to_string())
HError::Default("C_GenerateKeyPair not available on library".to_string())
})?(
self.session_handle(),
pMechanism,
@ -163,7 +163,7 @@ impl Session {
);
if rv != CKR_OK {
return Err(PError::Default(
return Err(HError::Default(
"Failed generating RSA key pair".to_string(),
));
}
@ -181,7 +181,7 @@ impl Session {
&self,
wrapping_key_handle: CK_OBJECT_HANDLE,
aes_key_handle: CK_OBJECT_HANDLE,
) -> PResult<Vec<u8>> {
) -> HResult<Vec<u8>> {
unsafe {
// Initialize the RSA-OAEP mechanism
let mut oaep_params = CK_RSA_PKCS_OAEP_PARAMS {
@ -203,7 +203,7 @@ impl Session {
let rv = self
.hsm()
.C_WrapKey
.ok_or_else(|| PError::Default("C_WrapKey not available on library".to_string()))?(
.ok_or_else(|| HError::Default("C_WrapKey not available on library".to_string()))?(
self.session_handle(),
&mut mechanism,
wrapping_key_handle,
@ -213,7 +213,7 @@ impl Session {
);
if rv != CKR_OK {
return Err(PError::Default(
return Err(HError::Default(
"Failed to get wrapped key length".to_string(),
));
}
@ -225,7 +225,7 @@ impl Session {
let rv = self
.hsm()
.C_WrapKey
.ok_or_else(|| PError::Default("C_WrapKey not available on library".to_string()))?(
.ok_or_else(|| HError::Default("C_WrapKey not available on library".to_string()))?(
self.session_handle(),
&mut mechanism,
wrapping_key_handle,
@ -235,7 +235,7 @@ impl Session {
);
if rv != CKR_OK {
return Err(PError::Default("Failed to wrap key".to_string()));
return Err(HError::Default("Failed to wrap key".to_string()));
}
// Truncate the buffer to the actual size of the wrapped key
@ -249,7 +249,7 @@ impl Session {
unwrapping_key_handle: CK_OBJECT_HANDLE,
wrapped_aes_key: &[u8],
aes_key_label: &str,
) -> PResult<CK_OBJECT_HANDLE> {
) -> HResult<CK_OBJECT_HANDLE> {
let mut wrapped_key = wrapped_aes_key.to_vec();
unsafe {
// Initialize the RSA-OAEP mechanism
@ -268,10 +268,56 @@ impl Session {
};
// Unwrap the key
let mut aes_key_template = aes_unwrap_key_template(aes_key_label);
let mut aes_key_template = [
CK_ATTRIBUTE {
type_: CKA_CLASS,
pValue: &CKO_SECRET_KEY as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_ULONG>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_KEY_TYPE,
pValue: &CKK_AES as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_ULONG>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_TOKEN,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_BBOOL>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_LABEL,
pValue: aes_key_label.as_ptr() as CK_VOID_PTR,
ulValueLen: aes_key_label.len() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_PRIVATE,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_BBOOL>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_SENSITIVE,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_BBOOL>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_EXTRACTABLE,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_BBOOL>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_ENCRYPT,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_BBOOL>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_DECRYPT,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_BBOOL>() as CK_ULONG,
},
];
let mut unwrapped_key_handle: CK_OBJECT_HANDLE = 0;
let rv = self.hsm().C_UnwrapKey.ok_or_else(|| {
PError::Default("C_UnwrapKey not available on library".to_string())
HError::Default("C_UnwrapKey not available on library".to_string())
})?(
self.session_handle(),
&mut mechanism,
@ -284,60 +330,10 @@ impl Session {
);
if rv != CKR_OK {
return Err(PError::Default("Failed to unwrap key".to_string()));
return Err(HError::Default("Failed to unwrap key".to_string()));
}
Ok(unwrapped_key_handle)
}
}
}
pub(crate) const fn aes_unwrap_key_template(label: &str) -> [CK_ATTRIBUTE; 9] {
[
CK_ATTRIBUTE {
type_: CKA_CLASS,
pValue: &CKO_SECRET_KEY as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_ULONG>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_KEY_TYPE,
pValue: &CKK_AES as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_ULONG>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_TOKEN,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_BBOOL>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_LABEL,
pValue: label.as_ptr() as CK_VOID_PTR,
ulValueLen: label.len() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_PRIVATE,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_BBOOL>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_SENSITIVE,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_BBOOL>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_EXTRACTABLE,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_BBOOL>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_ENCRYPT,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_BBOOL>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_DECRYPT,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_BBOOL>() as CK_ULONG,
},
]
}

381
crate/pkcs11/proteccio/src/session/session_impl.rs → crate/hsm/base_hsm/src/session/session_impl.rs
View file

@ -1,3 +1,40 @@
//! Hardware Security Module (HSM) Session Implementation
//!
//! This module provides the implementation of a session with a Hardware Security Module (HSM)
//! following the PKCS#11 standard. It includes functionality for:
//!
//! - Managing HSM session lifecycle (creation, authentication, closure)
//! - Object handling (creation, deletion, listing)
//! - Cryptographic operations (encryption, decryption)
//! - Key management (export, metadata retrieval)
//!
//! The implementation supports various cryptographic algorithms including:
//! - AES-GCM for symmetric encryption
//! - RSA PKCS#1 v1.5 and OAEP for asymmetric encryption
//!
//! # Key Features
//!
//! - Session management with HSM devices
//! - Object handle caching for improved performance
//! - Support for both symmetric and asymmetric cryptographic operations
//! - Key export capabilities with security controls
//! - Comprehensive error handling
//!
//! # Security Considerations
//!
//! - Sensitive key material is protected using the `Zeroizing` type
//! - Login state is tracked to ensure proper session closure
//! - Object handle caching is thread-safe using `Arc`
//!
//! # Examples
//!
//! ```no_run
//! use hsm::Session;
//!
//! let session = Session::new(hsm, session_handle, cache, true);
//! let random_bytes = session.generate_random(32)?;
//! ```
use std::{ptr, sync::Arc};
use cosmian_kms_interfaces::{
@ -5,32 +42,97 @@ use cosmian_kms_interfaces::{
KeyType, RsaPrivateKeyMaterial, RsaPublicKeyMaterial,
};
use pkcs11_sys::*;
use rand::{rngs::OsRng, TryRngCore};
use tracing::debug;
use zeroize::Zeroizing;
pub use crate::session::{aes::AesKeySize, rsa::RsaKeySize};
use crate::{
aes_mechanism, generate_random_nonce, rsa_mechanism, ObjectHandlesCache, PError, PResult,
};
use crate::{HError, HResult, ObjectHandlesCache};
pub enum ProteccioEncryptionAlgorithm {
/// Generate a random nonce of size T
/// This function is used to generate a random nonce for the AES GCM encryption
fn generate_random_nonce<const T: usize>() -> HResult<[u8; T]> {
let mut bytes = [0u8; T];
OsRng
.try_fill_bytes(&mut bytes)
.map_err(|e| HError::Default(format!("Error generating random nonce: {}", e)))?;
Ok(bytes)
}
/// Encryption algorithm supported by the HSM
pub enum HsmEncryptionAlgorithm {
AesGcm,
RsaPkcsV15,
RsaOaep,
}
impl From<CryptoAlgorithm> for ProteccioEncryptionAlgorithm {
impl From<CryptoAlgorithm> for HsmEncryptionAlgorithm {
fn from(algorithm: CryptoAlgorithm) -> Self {
match algorithm {
CryptoAlgorithm::AesGcm => ProteccioEncryptionAlgorithm::AesGcm,
CryptoAlgorithm::RsaPkcsV15 => ProteccioEncryptionAlgorithm::RsaPkcsV15,
CryptoAlgorithm::RsaOaep => ProteccioEncryptionAlgorithm::RsaOaep,
CryptoAlgorithm::AesGcm => HsmEncryptionAlgorithm::AesGcm,
CryptoAlgorithm::RsaPkcsV15 => HsmEncryptionAlgorithm::RsaPkcsV15,
CryptoAlgorithm::RsaOaep => HsmEncryptionAlgorithm::RsaOaep,
}
}
}
/// A session with an HSM (Hardware Security Module) that implements PKCS#11 interface.
/// This structure represents an active connection to the HSM and provides methods to
/// perform cryptographic operations and key management.
///
/// # Structure Fields
/// * `hsm` - Arc reference to the HSM library interface
/// * `session_handle` - PKCS#11 session handle
/// * `object_handles_cache` - Cache for object handles
/// * `is_logged_in` - Login state of the session
///
/// # Methods
/// The session provides several categories of operations:
///
/// ## Session Management
/// * `new()` - Creates a new session
/// * `close()` - Closes the session and logs out if necessary
///
/// ## Object Management
/// * `get_object_handle()` - Retrieves handle for an object by its ID
/// * `delete_object_handle()` - Removes an object handle from cache
/// * `list_objects()` - Lists objects matching specified filter
/// * `destroy_object()` - Deletes an object from the HSM
///
/// ## Cryptographic Operations
/// * `encrypt()` - Encrypts data using specified algorithm
/// * `decrypt()` - Decrypts data using specified algorithm
/// * `generate_random()` - Generates random data
///
/// ## Key Management
/// * `export_key()` - Exports a key from the HSM (if allowed)
/// * `get_key_metadata()` - Retrieves metadata about a key
/// * `get_key_type()` - Gets the type of a key
/// * `get_object_id()` - Gets the ID of an object
///
/// ## Internal Helpers
/// * `encrypt_with_mechanism()` - Internal encryption implementation
/// * `decrypt_with_mechanism()` - Internal decryption implementation
/// * `export_rsa_private_key()` - Exports RSA private key
/// * `export_rsa_public_key()` - Exports RSA public key
/// * `export_aes_key()` - Exports AES key
/// * `call_get_attributes()` - Helper for retrieving object attributes
///
/// # Safety
/// Many methods in this implementation contain unsafe blocks as they interact with
/// the PKCS#11 C interface. Care should be taken when using these methods, and all
/// preconditions must be met to ensure safe operation.
///
/// # Error Handling
/// Methods return `PResult<T>` which is a custom result type for handling HSM-related
/// errors. Operations can fail due to various reasons including:
/// * Invalid object handles
/// * Permission issues
/// * Communication errors with HSM
/// * Invalid parameters
/// * Unsupported operations
pub struct Session {
hsm: Arc<crate::proteccio::HsmLib>,
hsm: Arc<crate::hsm_lib::HsmLib>,
session_handle: CK_SESSION_HANDLE,
object_handles_cache: Arc<ObjectHandlesCache>,
is_logged_in: bool,
@ -38,7 +140,7 @@ pub struct Session {
impl Session {
pub fn new(
hsm: Arc<crate::proteccio::HsmLib>,
hsm: Arc<crate::hsm_lib::HsmLib>,
session_handle: CK_SESSION_HANDLE,
object_handles_cache: Arc<ObjectHandlesCache>,
is_logged_in: bool,
@ -51,44 +153,49 @@ impl Session {
}
}
pub(crate) fn hsm(&self) -> Arc<crate::proteccio::HsmLib> {
/// Get the HSM library interface
pub(crate) fn hsm(&self) -> Arc<crate::hsm_lib::HsmLib> {
self.hsm.clone()
}
/// Get the PKCS#11 session handle
pub(crate) fn session_handle(&self) -> CK_SESSION_HANDLE {
self.session_handle
}
/// Get the object handles cache
pub(crate) fn object_handles_cache(&self) -> Arc<ObjectHandlesCache> {
self.object_handles_cache.clone()
}
pub fn close(&self) -> PResult<()> {
/// Close the session and log out if necessary
pub fn close(&self) -> HResult<()> {
unsafe {
if self.is_logged_in {
let rv = self.hsm.C_Logout.ok_or_else(|| {
PError::Default("C_Logout not available on library".to_string())
HError::Default("C_Logout not available on library".to_string())
})?(self.session_handle);
if rv != CKR_OK {
return Err(PError::Default("Failed logging out".to_string()));
return Err(HError::Default("Failed logging out".to_string()));
}
}
let rv = self.hsm.C_CloseSession.ok_or_else(|| {
PError::Default("C_CloseSession not available on library".to_string())
HError::Default("C_CloseSession not available on library".to_string())
})?(self.session_handle);
if rv != CKR_OK {
return Err(PError::Default("Failed closing a session".to_string()));
return Err(HError::Default("Failed closing a session".to_string()));
}
Ok(())
}
}
pub fn get_object_handle(&self, object_id: &[u8]) -> PResult<CK_OBJECT_HANDLE> {
pub fn get_object_handle(&self, object_id: &[u8]) -> HResult<CK_OBJECT_HANDLE> {
if let Some(handle) = self.object_handles_cache.get(object_id) {
return Ok(handle);
}
// Proteccio does not allow the ID for secret keys so we use the label
// and we do the same on base HSM
let mut template = [CK_ATTRIBUTE {
type_: CKA_LABEL,
pValue: object_id.as_ptr() as CK_VOID_PTR,
@ -97,20 +204,20 @@ impl Session {
unsafe {
let rv = self.hsm.C_FindObjectsInit.ok_or_else(|| {
PError::Default("C_FindObjectsInit not available on library".to_string())
HError::Default("C_FindObjectsInit not available on library".to_string())
})?(
self.session_handle,
template.as_mut_ptr(),
template.len() as CK_ULONG,
);
if rv != CKR_OK {
return Err(PError::Default(format!("C_FindObjectsInit failed: {}", rv)));
return Err(HError::Default(format!("C_FindObjectsInit failed: {}", rv)));
}
let mut object_handle: CK_OBJECT_HANDLE = 0;
let mut object_count: CK_ULONG = 0;
let rv = self.hsm.C_FindObjects.ok_or_else(|| {
PError::Default("C_FindObjects not available on library".to_string())
HError::Default("C_FindObjects not available on library".to_string())
})?(
self.session_handle,
&mut object_handle,
@ -118,21 +225,21 @@ impl Session {
&mut object_count,
);
if rv != CKR_OK {
return Err(PError::Default(format!("C_FindObjects failed: {}", rv)));
return Err(HError::Default(format!("C_FindObjects failed: {}", rv)));
}
let rv = self.hsm.C_FindObjectsFinal.ok_or_else(|| {
PError::Default("C_FindObjectsFinal not available on library".to_string())
HError::Default("C_FindObjectsFinal not available on library".to_string())
})?(self.session_handle);
if rv != CKR_OK {
return Err(PError::Default(format!(
return Err(HError::Default(format!(
"C_FindObjectsFinal failed: {}",
rv
)));
}
if object_count == 0 {
return Err(PError::Default("Object not found".to_string()));
return Err(HError::Default("Object not found".to_string()));
}
//update cache
@ -147,7 +254,7 @@ impl Session {
self.object_handles_cache.remove(id)
}
pub fn generate_random(&self, len: usize) -> PResult<Vec<u8>> {
pub fn generate_random(&self, len: usize) -> HResult<Vec<u8>> {
unsafe {
let mut values = vec![0u8; len];
let values_ptr: *mut u8 = values.as_mut_ptr();
@ -156,16 +263,20 @@ impl Session {
#[cfg(not(target_os = "windows"))]
let len = u64::try_from(len)?;
let rv = self.hsm.C_GenerateRandom.ok_or_else(|| {
PError::Default("C_GenerateRandom not available on library".to_string())
HError::Default("C_GenerateRandom not available on library".to_string())
})?(self.session_handle, values_ptr, len);
if rv != CKR_OK {
return Err(PError::Default("Failed generating random data".to_string()));
return Err(HError::Default("Failed generating random data".to_string()));
}
Ok(values)
}
}
pub fn list_objects(&self, object_filter: HsmObjectFilter) -> PResult<Vec<CK_OBJECT_HANDLE>> {
/// List objects in the HSM that match the specified filter
/// The filter can be used to narrow down the search to specific types of objects
/// such as AES keys, RSA keys, etc.
/// If no filter is provided, all objects are listed.
pub fn list_objects(&self, object_filter: HsmObjectFilter) -> HResult<Vec<CK_OBJECT_HANDLE>> {
let mut object_handles: Vec<CK_OBJECT_HANDLE> = Vec::new();
let mut template: Vec<CK_ATTRIBUTE> = Vec::new();
match object_filter {
@ -217,14 +328,14 @@ impl Session {
unsafe {
let rv = self.hsm.C_FindObjectsInit.ok_or_else(|| {
PError::Default("C_FindObjectsInit not available on library".to_string())
HError::Default("C_FindObjectsInit not available on library".to_string())
})?(
self.session_handle,
template.as_mut_ptr(),
template.len() as CK_ULONG,
);
if rv != CKR_OK {
return Err(PError::Default(
return Err(HError::Default(
"Failed to initialize object search".to_string(),
));
}
@ -233,7 +344,7 @@ impl Session {
let mut object_count: CK_ULONG = 0;
loop {
let rv = self.hsm.C_FindObjects.ok_or_else(|| {
PError::Default("C_FindObjects not available on library".to_string())
HError::Default("C_FindObjects not available on library".to_string())
})?(
self.session_handle,
&mut object_handle,
@ -241,7 +352,7 @@ impl Session {
&mut object_count,
);
if rv != CKR_OK {
return Err(PError::Default("Failed to find objects".to_string()));
return Err(HError::Default("Failed to find objects".to_string()));
}
if object_count == 0 {
break;
@ -250,10 +361,10 @@ impl Session {
}
let rv = self.hsm.C_FindObjectsFinal.ok_or_else(|| {
PError::Default("C_FindObjectsFinal not available on library".to_string())
HError::Default("C_FindObjectsFinal not available on library".to_string())
})?(self.session_handle);
if rv != CKR_OK {
return Err(PError::Default(
return Err(HError::Default(
"Failed to finalize object search".to_string(),
));
}
@ -261,72 +372,144 @@ impl Session {
Ok(object_handles)
}
pub fn destroy_object(&self, object_handle: CK_OBJECT_HANDLE) -> PResult<()> {
/// Destroy an object in the HSM
pub fn destroy_object(&self, object_handle: CK_OBJECT_HANDLE) -> HResult<()> {
unsafe {
let rv = self.hsm.C_DestroyObject.ok_or_else(|| {
PError::Default("C_DestroyObject not available on library".to_string())
HError::Default("C_DestroyObject not available on library".to_string())
})?(self.session_handle, object_handle);
if rv != CKR_OK {
return Err(PError::Default("Failed to destroy object".to_string()));
return Err(HError::Default("Failed to destroy object".to_string()));
}
}
Ok(())
}
/// Encrypt data using the specified key and algorithm
pub fn encrypt(
&self,
key_handle: CK_OBJECT_HANDLE,
algorithm: ProteccioEncryptionAlgorithm,
algorithm: HsmEncryptionAlgorithm,
plaintext: &[u8],
) -> PResult<EncryptedContent> {
) -> HResult<EncryptedContent> {
Ok(match algorithm {
ProteccioEncryptionAlgorithm::AesGcm => {
HsmEncryptionAlgorithm::AesGcm => {
let mut nonce = generate_random_nonce::<12>()?;
let ciphertext = self.encrypt_with_mechanism(
key_handle,
&mut aes_mechanism!(&mut nonce),
plaintext,
)?;
let mut params = CK_AES_GCM_PARAMS {
pIv: &mut nonce as *mut u8,
ulIvLen: 12,
ulIvBits: 96,
pAAD: ptr::null_mut(),
ulAADLen: 0,
ulTagBits: 128,
};
let mut mechanism = CK_MECHANISM {
mechanism: CKM_AES_GCM,
pParameter: &mut params as *mut _ as CK_VOID_PTR,
ulParameterLen: size_of::<CK_AES_GCM_PARAMS>() as CK_ULONG,
};
let ciphertext =
self.encrypt_with_mechanism(key_handle, &mut mechanism, plaintext)?;
EncryptedContent {
iv: Some(nonce.to_vec()),
ciphertext: ciphertext[..ciphertext.len() - 16].to_vec(),
tag: Some(ciphertext[ciphertext.len() - 16..].to_vec()),
}
}
_ => EncryptedContent {
ciphertext: self.encrypt_with_mechanism(
key_handle,
&mut rsa_mechanism!(algorithm),
plaintext,
)?,
..Default::default()
},
HsmEncryptionAlgorithm::RsaPkcsV15 => {
let mut mechanism = CK_MECHANISM {
mechanism: CKM_RSA_PKCS,
pParameter: std::ptr::null_mut(),
ulParameterLen: 0,
};
EncryptedContent {
ciphertext: self.encrypt_with_mechanism(
key_handle,
&mut mechanism,
plaintext,
)?,
..Default::default()
}
}
HsmEncryptionAlgorithm::RsaOaep => {
let mut params = CK_RSA_PKCS_OAEP_PARAMS {
hashAlg: CKM_SHA256,
mgf: CKG_MGF1_SHA256,
source: CKZ_DATA_SPECIFIED,
pSourceData: std::ptr::null_mut(),
ulSourceDataLen: 0,
};
let mut mechanism = CK_MECHANISM {
mechanism: CKM_RSA_PKCS_OAEP,
pParameter: &mut params as *mut _ as CK_VOID_PTR,
ulParameterLen: std::mem::size_of::<CK_RSA_PKCS_OAEP_PARAMS>() as CK_ULONG,
};
EncryptedContent {
ciphertext: self.encrypt_with_mechanism(
key_handle,
&mut mechanism,
plaintext,
)?,
..Default::default()
}
}
})
}
/// Decrypt data using the specified key and algorithm
pub fn decrypt(
&self,
key_handle: CK_OBJECT_HANDLE,
algorithm: ProteccioEncryptionAlgorithm,
algorithm: HsmEncryptionAlgorithm,
ciphertext: &[u8],
) -> PResult<Zeroizing<Vec<u8>>> {
) -> HResult<Zeroizing<Vec<u8>>> {
match algorithm {
ProteccioEncryptionAlgorithm::AesGcm => {
HsmEncryptionAlgorithm::AesGcm => {
if ciphertext.len() < 12 {
return Err(PError::Default("Invalid AES GCM ciphertext".to_string()));
return Err(HError::Default("Invalid AES GCM ciphertext".to_string()));
}
let mut nonce: [u8; 12] = ciphertext[..12]
.try_into()
.map_err(|_| PError::Default("Invalid AES GCM nonce".to_string()))?;
let plaintext = self.decrypt_with_mechanism(
key_handle,
&mut aes_mechanism!(&mut nonce),
&ciphertext[12..],
)?;
.map_err(|_| HError::Default("Invalid AES GCM nonce".to_string()))?;
let mut params = CK_AES_GCM_PARAMS {
pIv: &mut nonce as *mut u8,
ulIvLen: 12,
ulIvBits: 96,
pAAD: ptr::null_mut(),
ulAADLen: 0,
ulTagBits: 128,
};
let mut mechanism = CK_MECHANISM {
mechanism: CKM_AES_GCM,
pParameter: &mut params as *mut _ as CK_VOID_PTR,
ulParameterLen: size_of::<CK_AES_GCM_PARAMS>() as CK_ULONG,
};
let plaintext =
self.decrypt_with_mechanism(key_handle, &mut mechanism, &ciphertext[12..])?;
Ok(plaintext)
}
_ => {
self.decrypt_with_mechanism(key_handle, &mut rsa_mechanism!(algorithm), ciphertext)
HsmEncryptionAlgorithm::RsaPkcsV15 => {
let mut mechanism = CK_MECHANISM {
mechanism: CKM_RSA_PKCS,
pParameter: std::ptr::null_mut(),
ulParameterLen: 0,
};
self.decrypt_with_mechanism(key_handle, &mut mechanism, ciphertext)
}
HsmEncryptionAlgorithm::RsaOaep => {
let mut params = CK_RSA_PKCS_OAEP_PARAMS {
hashAlg: CKM_SHA256,
mgf: CKG_MGF1_SHA256,
source: CKZ_DATA_SPECIFIED,
pSourceData: std::ptr::null_mut(),
ulSourceDataLen: 0,
};
let mut mechanism = CK_MECHANISM {
mechanism: CKM_RSA_PKCS_OAEP,
pParameter: &mut params as *mut _ as CK_VOID_PTR,
ulParameterLen: std::mem::size_of::<CK_RSA_PKCS_OAEP_PARAMS>() as CK_ULONG,
};
self.decrypt_with_mechanism(key_handle, &mut mechanism, ciphertext)
}
}
}
@ -336,16 +519,16 @@ impl Session {
key_handle: CK_OBJECT_HANDLE,
mechanism: &mut CK_MECHANISM,
data: &[u8],
) -> PResult<Vec<u8>> {
) -> HResult<Vec<u8>> {
let mut data = data.to_vec();
unsafe {
let ck_fn = self.hsm.C_EncryptInit.ok_or_else(|| {
PError::Default("C_EncryptInit not available on library".to_string())
HError::Default("C_EncryptInit not available on library".to_string())
})?;
let rv = ck_fn(self.session_handle, mechanism, key_handle);
if rv != CKR_OK {
return Err(PError::Default(
return Err(HError::Default(
"Failed to initialize encryption".to_string(),
));
}
@ -353,7 +536,7 @@ impl Session {
let ck_fn = self
.hsm
.C_Encrypt
.ok_or_else(|| PError::Default("C_Encrypt not available on library".to_string()))?;
.ok_or_else(|| HError::Default("C_Encrypt not available on library".to_string()))?;
let mut encrypted_data_len: CK_ULONG = 0;
let rv = ck_fn(
@ -364,7 +547,7 @@ impl Session {
&mut encrypted_data_len,
);
if rv != CKR_OK {
return Err(PError::Default(
return Err(HError::Default(
"Failed to get encrypted data length".to_string(),
));
}
@ -378,7 +561,7 @@ impl Session {
&mut encrypted_data_len,
);
if rv != CKR_OK {
return Err(PError::Default("Failed to encrypt data".to_string()));
return Err(HError::Default("Failed to encrypt data".to_string()));
}
encrypted_data.truncate(encrypted_data_len as usize);
@ -391,16 +574,16 @@ impl Session {
key_handle: CK_OBJECT_HANDLE,
mechanism: &mut CK_MECHANISM,
encrypted_data: &[u8],
) -> PResult<Zeroizing<Vec<u8>>> {
) -> HResult<Zeroizing<Vec<u8>>> {
let mut encrypted_data = encrypted_data.to_vec();
unsafe {
let ck_fn = self.hsm.C_DecryptInit.ok_or_else(|| {
PError::Default("C_DecryptInit not available on library".to_string())
HError::Default("C_DecryptInit not available on library".to_string())
})?;
let rv = ck_fn(self.session_handle, mechanism, key_handle);
if rv != CKR_OK {
return Err(PError::Default(
return Err(HError::Default(
"Failed to initialize decryption".to_string(),
));
}
@ -408,7 +591,7 @@ impl Session {
let ck_fn = self
.hsm
.C_Decrypt
.ok_or_else(|| PError::Default("C_Decrypt not available on library".to_string()))?;
.ok_or_else(|| HError::Default("C_Decrypt not available on library".to_string()))?;
let mut decrypted_data_len: CK_ULONG = 0;
let rv = ck_fn(
@ -419,7 +602,7 @@ impl Session {
&mut decrypted_data_len,
);
if rv != CKR_OK {
return Err(PError::Default(
return Err(HError::Default(
"Failed to get decrypted data length".to_string(),
));
}
@ -433,7 +616,7 @@ impl Session {
&mut decrypted_data_len,
);
if rv != CKR_OK {
return Err(PError::Default("Failed to decrypt data".to_string()));
return Err(HError::Default("Failed to decrypt data".to_string()));
}
decrypted_data.truncate(decrypted_data_len as usize);
@ -441,7 +624,8 @@ impl Session {
}
}
pub fn export_key(&self, key_handle: CK_OBJECT_HANDLE) -> PResult<Option<HsmObject>> {
/// Export a key from the HSM
pub fn export_key(&self, key_handle: CK_OBJECT_HANDLE) -> HResult<Option<HsmObject>> {
let mut key_type: CK_KEY_TYPE = CKK_VENDOR_DEFINED;
let mut class: CK_OBJECT_CLASS = CKO_VENDOR_DEFINED;
let mut template = [
@ -468,7 +652,7 @@ impl Session {
}
}
x => {
return Err(PError::Default(format!(
return Err(HError::Default(format!(
"Export: unsupported key type: {x}"
)));
}
@ -481,7 +665,7 @@ impl Session {
}
}
fn export_rsa_private_key(&self, key_handle: CK_OBJECT_HANDLE) -> PResult<Option<HsmObject>> {
fn export_rsa_private_key(&self, key_handle: CK_OBJECT_HANDLE) -> HResult<Option<HsmObject>> {
// Get the key size
let mut template = [
CK_ATTRIBUTE {
@ -608,7 +792,7 @@ impl Session {
return Ok(None);
}
let label = String::from_utf8(label_bytes)
.map_err(|e| PError::Default(format!("Failed to convert label to string: {}", e)))?;
.map_err(|e| HError::Default(format!("Failed to convert label to string: {}", e)))?;
Ok(Some(HsmObject::new(
KeyMaterial::RsaPrivateKey(RsaPrivateKeyMaterial {
modulus,
@ -624,7 +808,7 @@ impl Session {
)))
}
fn export_rsa_public_key(&self, key_handle: CK_OBJECT_HANDLE) -> PResult<Option<HsmObject>> {
fn export_rsa_public_key(&self, key_handle: CK_OBJECT_HANDLE) -> HResult<Option<HsmObject>> {
// Get the key size
let mut template = [
CK_ATTRIBUTE {
@ -679,7 +863,7 @@ impl Session {
return Ok(None);
}
let label = String::from_utf8(label_bytes)
.map_err(|e| PError::Default(format!("Failed to convert label to string: {}", e)))?;
.map_err(|e| HError::Default(format!("Failed to convert label to string: {}", e)))?;
Ok(Some(HsmObject::new(
KeyMaterial::RsaPublicKey(RsaPublicKeyMaterial {
modulus,
@ -689,7 +873,7 @@ impl Session {
)))
}
fn export_aes_key(&self, key_handle: CK_OBJECT_HANDLE) -> PResult<Option<HsmObject>> {
fn export_aes_key(&self, key_handle: CK_OBJECT_HANDLE) -> HResult<Option<HsmObject>> {
// Get the key size
let mut template = [
CK_ATTRIBUTE {
@ -739,7 +923,7 @@ impl Session {
return Ok(None);
}
let label = String::from_utf8(label_bytes)
.map_err(|e| PError::Default(format!("Failed to convert label to string: {}", e)))?;
.map_err(|e| HError::Default(format!("Failed to convert label to string: {}", e)))?;
Ok(Some(HsmObject::new(
KeyMaterial::AesKey(Zeroizing::new(key_value)),
label,
@ -750,12 +934,12 @@ impl Session {
&self,
key_handle: CK_OBJECT_HANDLE,
template: &mut [CK_ATTRIBUTE],
) -> PResult<Option<()>> {
) -> HResult<Option<()>> {
unsafe {
debug!("Retrieving Proteccio key attributes for key: {key_handle}");
// Get the length of the key value
let rv = self.hsm.C_GetAttributeValue.ok_or_else(|| {
PError::Default("C_GetAttributeValue not available on library".to_string())
HError::Default("C_GetAttributeValue not available on library".to_string())
})?(
self.session_handle,
key_handle,
@ -763,16 +947,16 @@ impl Session {
template.len() as CK_ULONG,
);
if rv == CKR_ATTRIBUTE_SENSITIVE {
return Err(PError::Default(format!(
"This key {key_handle} cannot be exported from the HSM."
)));
return Err(HError::Default(
"This key is sensitive and cannot be exported from the HSM.".to_string(),
));
}
if rv == CKR_OBJECT_HANDLE_INVALID {
// The key was not found
return Ok(None);
}
if rv != CKR_OK {
return Err(PError::Default(format!(
return Err(HError::Default(format!(
"Failed to get the HSM attributes for key {key_handle}"
)));
}
@ -780,7 +964,8 @@ impl Session {
}
}
pub fn get_key_metadata(&self, key_handle: CK_OBJECT_HANDLE) -> PResult<Option<KeyMetadata>> {
/// Get the metadata for a key
pub fn get_key_metadata(&self, key_handle: CK_OBJECT_HANDLE) -> HResult<Option<KeyMetadata>> {
let key_type = match self.get_key_type(key_handle)? {
None => return Ok(None),
Some(key_type) => key_type,
@ -830,13 +1015,13 @@ impl Session {
return Ok(None);
}
String::from_utf8(label_bytes).map_err(|e| {
PError::Default(format!("Failed to convert label to string: {}", e))
HError::Default(format!("Failed to convert label to string: {}", e))
})?
};
Ok(Some(KeyMetadata {
key_type,
key_length_in_bits: usize::try_from(key_size).map_err(|e| {
PError::Default(format!("Failed to convert key size to usize: {}", e))
HError::Default(format!("Failed to convert key size to usize: {}", e))
})? * 8,
sensitive: sensitive == CK_TRUE,
id: label,
@ -890,7 +1075,7 @@ impl Session {
String::new()
} else {
String::from_utf8(label_bytes).map_err(|e| {
PError::Default(format!("Failed to convert label to string: {}", e))
HError::Default(format!("Failed to convert label to string: {}", e))
})?
};
let sensitive = sensitive == CK_TRUE;
@ -909,7 +1094,7 @@ impl Session {
/// * `key_handle` - The key handle
/// # Returns
/// * `Result<Option<KeyType>>` - The key type if the key exists
pub(crate) fn get_key_type(&self, key_handle: CK_OBJECT_HANDLE) -> PResult<Option<KeyType>> {
pub fn get_key_type(&self, key_handle: CK_OBJECT_HANDLE) -> HResult<Option<KeyType>> {
let mut key_type: CK_KEY_TYPE = CKK_VENDOR_DEFINED;
let mut class: CK_OBJECT_CLASS = CKO_VENDOR_DEFINED;
let mut template = [
@ -941,7 +1126,7 @@ impl Session {
}
}
x => {
return Err(PError::Default(format!(
return Err(HError::Default(format!(
"Export: unsupported key type: {x}"
)));
}
@ -957,7 +1142,7 @@ impl Session {
pub(crate) fn get_object_id(
&self,
object_handle: CK_OBJECT_HANDLE,
) -> PResult<Option<Vec<u8>>> {
) -> HResult<Option<Vec<u8>>> {
let mut template = [CK_ATTRIBUTE {
type_: CKA_ID,
pValue: ptr::null_mut(),

204
crate/hsm/base_hsm/src/slots.rs Normal file
View file

@ -0,0 +1,204 @@
use std::{
num::NonZeroUsize,
ptr,
sync::{Arc, Mutex},
};
use lru::LruCache;
use pkcs11_sys::{
CKF_RW_SESSION, CKF_SERIAL_SESSION, CKR_OK, CKR_USER_ALREADY_LOGGED_IN, CKU_USER, CK_FLAGS,
CK_OBJECT_HANDLE, CK_SESSION_HANDLE, CK_SLOT_ID, CK_ULONG, CK_UTF8CHAR_PTR,
};
use tracing::warn;
use crate::{hsm_lib::HsmLib, HError, HResult, Session};
/// A cache structure that maps byte vectors to CK_OBJECT_HANDLE values using an LRU (Least Recently Used) strategy.
///
/// This struct wraps a mutex-protected LRU cache that associates object identifiers (as byte vectors)
/// with their corresponding PKCS#11 object handles. The cache helps improve performance by reducing
/// repeated lookups of frequently used objects.
///
/// The LRU cache automatically removes the least recently accessed entries when it reaches its capacity,
/// helping to manage memory usage while maintaining quick access to frequently used handles.
pub struct ObjectHandlesCache(Mutex<LruCache<Vec<u8>, CK_OBJECT_HANDLE>>);
impl Default for ObjectHandlesCache {
fn default() -> Self {
Self::new()
}
}
impl ObjectHandlesCache {
pub fn new() -> Self {
#[allow(unsafe_code)]
let max = unsafe { NonZeroUsize::new_unchecked(100) };
ObjectHandlesCache(Mutex::new(LruCache::new(max)))
}
/// Get the object handle for the specified key.
pub fn get(&self, key: &[u8]) -> Option<CK_OBJECT_HANDLE> {
self.0
.lock()
.expect("HSM: failed to lock the handles cache")
.get(key)
.copied()
}
/// Insert a new object handle into the cache.
pub fn insert(&self, key: Vec<u8>, value: CK_OBJECT_HANDLE) {
self.0
.lock()
.expect("HSM: failed to lock the handles cache")
.put(key, value);
}
/// Remove an object handle from the cache.
pub fn remove(&self, key: &[u8]) {
self.0
.lock()
.expect("HSM: failed to lock the handles cache")
.pop(key);
}
}
/// A manager for a specific PKCS#11 slot in a Hardware Security Module (HSM).
///
/// This structure maintains the connection to a specific slot within an HSM,
/// managing the slot's session and object handles.
///
/// # Fields
///
/// * `hsm_lib` - A thread-safe reference to the HSM library interface
/// * `slot_id` - The unique identifier for this HSM slot
/// * `object_handles_cache` - A thread-safe cache of object handles for this slot
/// * `_login_session` - An optional authenticated session with the HSM slot
///
/// The SlotManager is responsible for coordinating operations on a specific HSM slot,
/// including session management and object handle caching.
pub struct SlotManager {
hsm_lib: Arc<HsmLib>,
slot_id: usize,
object_handles_cache: Arc<ObjectHandlesCache>,
_login_session: Option<Session>,
}
impl SlotManager {
/// Create a new SlotManager instance for the specified slot.
/// If a login password is provided, the slot will be authenticated with the HSM.
/// # Arguments
/// * `hsm_lib` - A thread-safe reference to the HSM library interface
/// * `slot_id` - The unique identifier for this HSM slot
/// * `login_password` - An optional password to authenticate the slot
/// # Returns
/// * `PResult<SlotManager>` - A result containing the SlotManager instance
/// # Errors
/// * If the slot cannot be opened or authenticated, an error is returned
/// * If the HSM library does not support the necessary functions, an error is returned
/// * If the HSM returns an error during session creation or login, an error is returned
/// # Safety
/// This function calls unsafe FFI functions from the HSM library to open a session and authenticate the slot.
/// The function is safe to call, but care must be taken when using the resulting SlotManager instance.
pub fn instantiate(
hsm_lib: Arc<HsmLib>,
slot_id: usize,
login_password: Option<String>,
) -> HResult<Self> {
let object_handles_cache = Arc::new(ObjectHandlesCache::new());
if let Some(password) = login_password {
let login_session = Self::open_session_(
&hsm_lib,
slot_id,
false,
object_handles_cache.clone(),
Some(password),
)?;
Ok(SlotManager {
hsm_lib,
slot_id,
object_handles_cache,
_login_session: Some(login_session),
})
} else {
Ok(SlotManager {
hsm_lib,
slot_id,
object_handles_cache,
_login_session: None,
})
}
}
/// Open a new session with the HSM slot.
/// The session can be read-only or read-write, depending on the `read_write` parameter.
/// # Arguments
/// * `read_write` - A boolean flag indicating whether the session should be read-write
/// # Returns
/// * `PResult<Session>` - A result containing the new session instance
/// # Errors
/// * If the session cannot be opened, an error is returned
/// * If the HSM library does not support the necessary functions, an error is returned
/// * If the HSM returns an error during session creation, an error is returned
/// # Safety
/// This function calls unsafe FFI functions from the HSM library to open a session.
/// The function is safe to call, but care must be taken when using the resulting Session instance.
/// The session is automatically closed when the Session instance is dropped.
pub fn open_session(&self, read_write: bool) -> HResult<Session> {
Self::open_session_(
&self.hsm_lib,
self.slot_id,
read_write,
self.object_handles_cache.clone(),
None,
)
}
fn open_session_(
hsm_lib: &Arc<HsmLib>,
slot_id: usize,
read_write: bool,
object_handles_cache: Arc<ObjectHandlesCache>,
login_password: Option<String>,
) -> HResult<Session> {
let slot_id: CK_SLOT_ID = slot_id as CK_SLOT_ID;
let flags: CK_FLAGS = if read_write {
CKF_RW_SESSION | CKF_SERIAL_SESSION
} else {
CKF_SERIAL_SESSION
};
let mut session_handle: CK_SESSION_HANDLE = 0;
unsafe {
let rv = hsm_lib.C_OpenSession.ok_or_else(|| {
HError::Default("C_OpenSession not available on library".to_string())
})?(slot_id, flags, ptr::null_mut(), None, &mut session_handle);
if rv != CKR_OK {
return Err(HError::Default(format!(
"HSM: Failed opening a session: {rv}å"
)));
}
if let Some(password) = login_password.as_ref() {
let mut pwd_bytes = password.as_bytes().to_vec();
let rv = hsm_lib.C_Login.ok_or_else(|| {
HError::Default("C_Login not available on library".to_string())
})?(
session_handle,
CKU_USER,
pwd_bytes.as_mut_ptr() as CK_UTF8CHAR_PTR,
pwd_bytes.len() as CK_ULONG,
);
if rv == CKR_USER_ALREADY_LOGGED_IN {
warn!("user already logged in, ignoring logging");
} else if rv != CKR_OK {
return Err(HError::Default("Failed logging in".to_string()));
}
}
Ok(Session::new(
hsm_lib.clone(),
session_handle,
object_handles_cache,
login_password.is_some(),
))
}
}
}

31
crate/hsm/base_hsm/src/test_helpers.rs Normal file
View file

@ -0,0 +1,31 @@
use std::sync::Once;
use tracing::Level;
use tracing_subscriber::FmtSubscriber;
use crate::{HError, HResult};
static TRACING_INIT: Once = Once::new();
pub fn initialize_logging() {
TRACING_INIT.call_once(|| {
let subscriber = FmtSubscriber::builder()
.with_max_level(Level::INFO) // Adjust the level as needed
.with_writer(std::io::stdout)
.finish();
tracing::subscriber::set_global_default(subscriber)
.expect("Setting default subscriber failed");
});
}
pub fn get_hsm_password() -> HResult<String> {
let user_password = option_env!("HSM_USER_PASSWORD")
.ok_or_else(|| {
HError::Default(
"The user password for the HSM is not set. Please set the HSM_USER_PASSWORD \
environment variable"
.to_string(),
)
})?
.to_string();
Ok(user_password)
}

27
crate/hsm/proteccio/Cargo.toml Normal file
View file

@ -0,0 +1,27 @@
[package]
name = "proteccio_pkcs11_loader"
version.workspace = true
authors.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
rust-version.workspace = true
description = "Proteccio HSM PKCS#11 loader"
[lib]
doctest = false
[dependencies]
cosmian_kms_base_hsm = { path = "../base_hsm" }
cosmian_kms_interfaces = { path = "../../interfaces" }
libloading = { workspace = true }
pkcs11_sys = { path = "../../pkcs11/sys" }
tracing = { workspace = true }
uuid = { workspace = true, features = ["v4"] }
[dev-dependencies]
uuid = { workspace = true }
[features]
# Enable this feature to run Proteccio tests (they require a Proteccio HSM)
proteccio = []

20
crate/pkcs11/proteccio/README.md → crate/hsm/proteccio/README.md
View file

@ -1,16 +1,20 @@
# Proteccio HSM wrapper
This is a wrapper for the Proteccio HSM library. It is written in C++ and provides a simple interface to the Proteccio
This is a wrapper for the Proteccio HSM library. It is written in Rust and provides a simple interface to the Proteccio
HSM library.
## Installation
The library is installed at `/lib/libnethsm.so`
The library must be installed at `/lib/libnethsm.so`
The configuration file is at `/etc/proteccio/proteccio.rc`
The log file and log level are specified in this file.
All other files shouold go to `/etgc/proteccio`
to view the logs use the command `tail -f /var/log/proteccio.log`
- `proteccio.rc` is the configuration file
- `proteccio.crt` is the certificate file of the (net) HSM
- `proteccio_client.key` and `proteccio_client.crt` are the client certificate and key for the HSM
The log file and log level are specified in the `proteccio.rc` files.
To view the logs use the command `tail -f /var/log/proteccio.log`
To verify the configuration:
@ -39,3 +43,9 @@ Serial Number: 81610-0040000161
Label: HSM1-V1
...
```
To list tokens in a slot:
```bash
nethsmtool -l <slot_id> <slot_password>
```

8
crate/hsm/proteccio/src/lib.rs Normal file
View file

@ -0,0 +1,8 @@
//! Copyright 2024 Cosmian Tech SAS
use cosmian_kms_base_hsm::BaseHsm;
pub type Proteccio = BaseHsm;
#[cfg(test)]
#[cfg(feature = "proteccio")]
mod tests;

453
crate/hsm/proteccio/src/tests.rs Normal file
View file

@ -0,0 +1,453 @@
//! These tests require a connection to a working HSM and are gated behind the `proteccio` feature.
//! To run a test, cd into the crate directory and run (replace `XXX` with the actual password).
//! The tests should be run in release mode as the behaviour of the PKCS#11 library is different in debug mode.
//! ```
//! HSM_USER_PASSWORD=XXX cargo test --release --target x86_64-unknown-linux-gnu --features proteccio -- tests::test_all
//! ```
use std::{collections::HashMap, ptr, sync::Arc, thread};
use cosmian_kms_base_hsm::{
test_helpers::{get_hsm_password, initialize_logging},
AesKeySize, HError, HResult, HsmEncryptionAlgorithm, RsaKeySize, SlotManager,
};
use cosmian_kms_interfaces::{HsmObjectFilter, KeyMaterial, KeyType};
use libloading::Library;
use pkcs11_sys::{CKF_OS_LOCKING_OK, CKR_OK, CK_C_INITIALIZE_ARGS, CK_RV, CK_VOID_PTR};
use tracing::info;
use uuid::Uuid;
use crate::Proteccio;
fn get_slot() -> HResult<Arc<SlotManager>> {
let user_password = get_hsm_password()?;
let passwords = HashMap::from([(0x04, Some(user_password.clone()))]);
let hsm = Proteccio::instantiate("/lib/libnethsm.so", passwords)?;
let manager = hsm.get_slot(0x04)?;
Ok(manager)
}
#[test]
fn test_all() -> HResult<()> {
test_hsm_get_info()?;
test_destroy_all()?;
test_generate_aes_key()?;
test_generate_rsa_keypair()?;
test_rsa_key_wrap()?;
test_rsa_pkcs_encrypt()?;
test_rsa_oaep_encrypt()?;
test_aes_gcm_encrypt()?;
multi_threaded_rsa_encrypt_decrypt_test()?;
test_get_key_metadata()?;
test_list_objects()?;
test_destroy_all()?;
Ok(())
}
#[test]
fn low_level_test() -> HResult<()> {
let path = "/lib/libnethsm.so";
let library = unsafe { Library::new(path) }?;
let init = unsafe { library.get::<fn(p_init_args: CK_VOID_PTR) -> CK_RV>(b"C_Initialize") }?;
let mut p_init_args = CK_C_INITIALIZE_ARGS {
CreateMutex: None,
DestroyMutex: None,
LockMutex: None,
UnlockMutex: None,
flags: CKF_OS_LOCKING_OK,
pReserved: ptr::null_mut(),
};
let rv = init(&mut p_init_args as *const CK_C_INITIALIZE_ARGS as CK_VOID_PTR);
assert_eq!(rv, CKR_OK);
Ok(())
}
#[test]
fn test_hsm_get_info() -> HResult<()> {
initialize_logging();
let hsm = Proteccio::instantiate("/lib/libnethsm.so", HashMap::new())?;
let info = hsm.get_info()?;
info!("Connected to the HSM: {info}");
Ok(())
}
#[test]
fn test_generate_aes_key() -> HResult<()> {
initialize_logging();
let key_id = Uuid::new_v4().to_string();
let slot = get_slot()?;
let session = slot.open_session(true)?;
let key_handle = session.generate_aes_key(key_id.as_bytes(), AesKeySize::Aes256, false)?;
info!("Generated exportable AES key: {}", key_id);
// assert the key handles are identical
assert_eq!(key_handle, session.get_object_handle(key_id.as_bytes())?);
// re-export the key
let key = session
.export_key(key_handle)?
.expect("Failed to find the key");
let key_bytes = match key.key_material() {
KeyMaterial::AesKey(v) => v,
KeyMaterial::RsaPrivateKey(_) | KeyMaterial::RsaPublicKey(_) => {
panic!("Expected an AES key");
}
};
assert_eq!(key_bytes.len() * 8, 256);
assert_eq!(key.id(), key_id.as_str());
match key.key_material() {
KeyMaterial::AesKey(v) => {
assert_eq!(v.len(), 32);
}
KeyMaterial::RsaPrivateKey(_) | KeyMaterial::RsaPublicKey(_) => {
panic!("Expected an AES key");
}
}
// Generate a sensitive AES key
let key_id = Uuid::new_v4().to_string();
let key_handle = session.generate_aes_key(key_id.as_bytes(), AesKeySize::Aes256, true)?;
info!("Generated sensitive AES key: {}", key_id);
// assert the key handles are identical
assert_eq!(key_handle, session.get_object_handle(key_id.as_bytes())?);
// it should not be exportable
assert!(session.export_key(key_handle).is_err());
Ok(())
}
#[test]
fn test_generate_rsa_keypair() -> HResult<()> {
initialize_logging();
let sk_id = Uuid::new_v4().to_string();
let pk_id = sk_id.clone() + "_pk ";
let slot = get_slot()?;
let session = slot.open_session(true)?;
let (sk_handle, pk_handle) = session.generate_rsa_key_pair(
sk_id.as_bytes(),
pk_id.as_bytes(),
RsaKeySize::Rsa2048,
false,
)?;
info!("Generated exportable RSA key: sk: {sk_id}, pk: {pk_id}");
// export the private key
assert_eq!(sk_handle, session.get_object_handle(sk_id.as_bytes())?);
let key = session
.export_key(sk_handle)?
.expect("Failed to find the private key");
assert_eq!(key.id(), sk_id.as_str());
match key.key_material() {
KeyMaterial::RsaPrivateKey(v) => {
assert_eq!(v.modulus.len() * 8, 2048);
}
KeyMaterial::RsaPublicKey(_) | KeyMaterial::AesKey(_) => {
panic!("Expected an RSA private key");
}
}
// export the public key
assert_eq!(pk_handle, session.get_object_handle(pk_id.as_bytes())?);
let key = session
.export_key(pk_handle)?
.expect("Failed to find the public key");
assert_eq!(key.id(), pk_id.as_str());
match key.key_material() {
KeyMaterial::RsaPublicKey(v) => {
assert_eq!(v.modulus.len() * 8, 2048);
}
KeyMaterial::RsaPrivateKey(_) | KeyMaterial::AesKey(_) => {
panic!("Expected an RSA public key");
}
}
// Generate a sensitive AES key
let sk_id = Uuid::new_v4().to_string();
let pk_id = sk_id.clone() + "_pk ";
session.generate_rsa_key_pair(
sk_id.as_bytes(),
pk_id.as_bytes(),
RsaKeySize::Rsa2048,
true,
)?;
info!("Generated sensitive RSA key: sk: {sk_id}, pk: {pk_id}");
// the private key should not be exportable
let sk_handle = session.get_object_handle(sk_id.as_bytes())?;
assert!(session.export_key(sk_handle).is_err());
// the public key should be exportable
let pk_handle = session.get_object_handle(pk_id.as_bytes())?;
let _key = session.export_key(pk_handle)?;
Ok(())
}
#[test]
fn test_rsa_key_wrap() -> HResult<()> {
initialize_logging();
let key_id = Uuid::new_v4().to_string();
let slot = get_slot()?;
let session = slot.open_session(true)?;
let symmetric_key = session.generate_aes_key(key_id.as_bytes(), AesKeySize::Aes256, true)?;
let sk_id = Uuid::new_v4().to_string();
let pk_id = sk_id.clone() + "_pk ";
let (sk, pk) = session.generate_rsa_key_pair(
sk_id.as_bytes(),
pk_id.as_bytes(),
RsaKeySize::Rsa2048,
true,
)?;
let encrypted_key = session.wrap_aes_key_with_rsa_oaep(pk, symmetric_key)?;
assert_eq!(encrypted_key.len(), 2048 / 8);
let decrypted_key =
session.unwrap_aes_key_with_rsa_oaep(sk, &encrypted_key, "another_label")?;
info!("Unwrapped symmetric key with handle: {}", decrypted_key);
Ok(())
}
#[test]
fn test_rsa_pkcs_encrypt() -> HResult<()> {
initialize_logging();
let slot = get_slot()?;
let session = slot.open_session(true)?;
let data = b"Hello, World!";
let sk_id = Uuid::new_v4().to_string();
let pk_id = sk_id.clone() + "_pk ";
let (sk, pk) = session.generate_rsa_key_pair(
sk_id.as_bytes(),
pk_id.as_bytes(),
RsaKeySize::Rsa2048,
true,
)?;
let enc = session.encrypt(pk, HsmEncryptionAlgorithm::RsaPkcsV15, data)?;
assert_eq!(enc.ciphertext.len(), 2048 / 8);
let plaintext = session.decrypt(sk, HsmEncryptionAlgorithm::RsaPkcsV15, &enc.ciphertext)?;
assert_eq!(plaintext.as_slice(), data);
info!("Successfully encrypted/decrypted with RSA PKCS");
Ok(())
}
#[test]
fn test_rsa_oaep_encrypt() -> HResult<()> {
initialize_logging();
let slot = get_slot()?;
let session = slot.open_session(true)?;
let data = b"Hello, World!";
let sk_id = Uuid::new_v4().to_string();
let pk_id = sk_id.clone() + "_pk ";
let (sk, pk) = session.generate_rsa_key_pair(
sk_id.as_bytes(),
pk_id.as_bytes(),
RsaKeySize::Rsa2048,
true,
)?;
let enc = session.encrypt(pk, HsmEncryptionAlgorithm::RsaOaep, data)?;
assert_eq!(enc.ciphertext.len(), 2048 / 8);
let plaintext = session.decrypt(sk, HsmEncryptionAlgorithm::RsaOaep, &enc.ciphertext)?;
assert_eq!(plaintext.as_slice(), data);
info!("Successfully encrypted/decrypted with AES OAEP");
Ok(())
}
#[test]
fn test_aes_gcm_encrypt() -> HResult<()> {
initialize_logging();
let slot = get_slot()?;
let session = slot.open_session(true)?;
let data = b"Hello, World!";
let key_id = Uuid::new_v4().to_string();
let sk = session.generate_aes_key(key_id.as_bytes(), AesKeySize::Aes256, true)?;
let enc = session.encrypt(sk, HsmEncryptionAlgorithm::AesGcm, data)?;
assert_eq!(enc.ciphertext.len(), data.len());
assert_eq!(enc.tag.clone().unwrap_or_default().len(), 16);
assert_eq!(enc.iv.clone().unwrap_or_default().len(), 12);
let plaintext = session.decrypt(
sk,
HsmEncryptionAlgorithm::AesGcm,
[
enc.iv.unwrap_or_default(),
enc.ciphertext,
enc.tag.unwrap_or_default(),
]
.concat()
.as_slice(),
)?;
assert_eq!(plaintext.as_slice(), data);
info!("Successfully encrypted/decrypted with AES GCM");
Ok(())
}
#[test]
fn multi_threaded_rsa_encrypt_decrypt_test() -> HResult<()> {
initialize_logging();
// Initialize the HSM once and share it across threads
let slot = get_slot()?;
let mut handles = vec![];
for _ in 0..4 {
let slot = slot.clone();
let handle = thread::spawn(move || {
let session = slot.open_session(true)?;
let data = b"Hello, World!";
let sk_id = Uuid::new_v4().to_string();
let pk_id = sk_id.clone() + "_pk ";
let (sk, pk) = session.generate_rsa_key_pair(
sk_id.as_bytes(),
pk_id.as_bytes(),
RsaKeySize::Rsa2048,
true,
)?;
let encrypted_content = session.encrypt(pk, HsmEncryptionAlgorithm::RsaOaep, data)?;
assert_eq!(encrypted_content.ciphertext.len(), 2048 / 8);
let plaintext = session.decrypt(
sk,
HsmEncryptionAlgorithm::RsaOaep,
&encrypted_content.ciphertext,
)?;
assert_eq!(plaintext.as_slice(), data);
Ok::<(), HError>(())
});
handles.push(handle);
}
for handle in handles {
handle.join().expect("Thread panicked")?;
}
info!("Successfully encrypted/decrypted with RSA OAEP in multiple threads");
Ok(())
}
#[test]
fn test_list_objects() -> HResult<()> {
initialize_logging();
let slot = get_slot()?;
let session = slot.open_session(true)?;
let objects = session.list_objects(HsmObjectFilter::Any)?;
for object in objects.iter() {
session.destroy_object(*object)?;
}
let objects = session.list_objects(HsmObjectFilter::Any)?;
assert_eq!(objects.len(), 0);
let sk_id = Uuid::new_v4().to_string();
let pk_id = sk_id.clone() + "_pk ";
let (_sk, _pk) = session.generate_rsa_key_pair(
sk_id.as_bytes(),
pk_id.as_bytes(),
RsaKeySize::Rsa2048,
false,
)?;
let objects = session.list_objects(HsmObjectFilter::Any)?;
assert_eq!(objects.len(), 2);
let objects = session.list_objects(HsmObjectFilter::RsaKey)?;
assert_eq!(objects.len(), 2);
let objects = session.list_objects(HsmObjectFilter::RsaPublicKey)?;
assert_eq!(objects.len(), 1);
let objects = session.list_objects(HsmObjectFilter::RsaPrivateKey)?;
assert_eq!(objects.len(), 1);
let objects = session.list_objects(HsmObjectFilter::AesKey)?;
assert_eq!(objects.len(), 0);
// add another keypair
let sk_id = Uuid::new_v4().to_string();
let pk_id = sk_id.clone() + "_pk ";
let (_sk, _pk) = session.generate_rsa_key_pair(
sk_id.as_bytes(),
pk_id.as_bytes(),
RsaKeySize::Rsa3072,
false,
)?;
let objects = session.list_objects(HsmObjectFilter::Any)?;
assert_eq!(objects.len(), 4);
let objects = session.list_objects(HsmObjectFilter::RsaKey)?;
assert_eq!(objects.len(), 4);
let objects = session.list_objects(HsmObjectFilter::RsaPublicKey)?;
assert_eq!(objects.len(), 2);
let objects = session.list_objects(HsmObjectFilter::RsaPrivateKey)?;
assert_eq!(objects.len(), 2);
let objects = session.list_objects(HsmObjectFilter::AesKey)?;
assert_eq!(objects.len(), 0);
// add an AES key
let key_id = Uuid::new_v4().to_string();
let _key = session.generate_aes_key(key_id.as_bytes(), AesKeySize::Aes256, false)?;
let objects = session.list_objects(HsmObjectFilter::Any)?;
assert_eq!(objects.len(), 5);
let objects = session.list_objects(HsmObjectFilter::AesKey)?;
assert_eq!(objects.len(), 1);
info!("Listed all objects");
Ok(())
}
#[test]
fn test_get_key_metadata() -> HResult<()> {
initialize_logging();
let slot = get_slot()?;
let session = slot.open_session(true)?;
// generate an AES key
let key_id = Uuid::new_v4().to_string();
let key_handle = session.generate_aes_key(key_id.as_bytes(), AesKeySize::Aes256, true)?;
// get the key basics
let key_type = session
.get_key_type(key_handle)?
.ok_or_else(|| HError::Default("Key not found".to_string()))?;
assert_eq!(key_type, KeyType::AesKey);
// get the metadata
let metadata = session
.get_key_metadata(key_handle)?
.ok_or_else(|| HError::Default("Key not found".to_string()))?;
assert_eq!(metadata.key_type, KeyType::AesKey);
assert!(metadata.sensitive);
assert_eq!(metadata.key_length_in_bits, 256);
assert_eq!(metadata.id.as_str(), key_id.as_str());
// generate an RSA keypair
let sk_id = Uuid::new_v4().to_string();
let pk_id = sk_id.clone() + "_pk ";
let (sk, pk) = session.generate_rsa_key_pair(
sk_id.as_bytes(),
pk_id.as_bytes(),
RsaKeySize::Rsa2048,
true,
)?;
// get the private key basics
let key_type = session
.get_key_type(sk)?
.ok_or_else(|| HError::Default("Key not found".to_string()))?;
assert_eq!(key_type, KeyType::RsaPrivateKey);
// get the private key metadata
let metadata = session
.get_key_metadata(sk)?
.ok_or_else(|| HError::Default("Key not found".to_string()))?;
assert_eq!(metadata.key_type, KeyType::RsaPrivateKey);
assert_eq!(metadata.key_length_in_bits, 2048);
assert_eq!(metadata.id.as_str(), sk_id.as_str());
assert!(metadata.sensitive);
// get the public key basics
let key_type = session
.get_key_type(pk)?
.ok_or_else(|| HError::Default("Key not found".to_string()))?;
assert_eq!(key_type, KeyType::RsaPublicKey);
// get the public key metadata
let metadata = session
.get_key_metadata(pk)?
.ok_or_else(|| HError::Default("Key not found".to_string()))?;
assert_eq!(metadata.key_type, KeyType::RsaPublicKey);
// assert!(metadata.sensitive);
assert_eq!(metadata.key_length_in_bits, 2048);
assert_eq!(metadata.id.as_str(), pk_id.as_str());
info!("Got key metadata");
Ok(())
}
#[test]
fn test_destroy_all() -> HResult<()> {
initialize_logging();
let slot = get_slot()?;
let session = slot.open_session(true)?;
let objects = session.list_objects(HsmObjectFilter::Any)?;
for object in objects.iter() {
session.destroy_object(*object)?;
}
let objects = session.list_objects(HsmObjectFilter::Any)?;
assert_eq!(objects.len(), 0);
info!("Destroyed all objects");
Ok(())
}

28
crate/hsm/utimaco/Cargo.toml Normal file
View file

@ -0,0 +1,28 @@
[package]
name = "utimaco_pkcs11_loader"
version.workspace = true
authors.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
rust-version.workspace = true
description = "utimaco HSM PKCS#11 loader"
[lib]
doctest = false
[dependencies]
cosmian_kms_base_hsm = { path = "../base_hsm" }
cosmian_kms_interfaces = { path = "../../interfaces" }
libloading = { workspace = true }
pkcs11_sys = { path = "../../pkcs11/sys" }
tracing = { workspace = true }
uuid = { workspace = true, features = ["v4"] }
[dev-dependencies]
tracing-subscriber = { workspace = true, features = ["env-filter"] }
uuid = { workspace = true }
[features]
# Enable this feature to run utimaco tests (they require a utimaco HSM)
utimaco = []

249
crate/hsm/utimaco/README.md Normal file
View file

@ -0,0 +1,249 @@
# Utimaco HSM
<!-- TOC -->
- [Utimaco HSM](#utimaco-hsm)
- [Installing the simulator](#installing-the-simulator)
- [ARM](#arm)
- [AMD64](#amd64)
- [Download and run the simulator](#download-and-run-the-simulator)
- [Configure the PKCS#11 connection on the KMS server](#configure-the-pkcs11-connection-on-the-kms-server)
- [PKCS#11 library](#pkcs11-library)
- [Configuration file](#configuration-file)
- [Test the PKCS#11 configuration](#test-the-pkcs11-configuration)
- [When a bridged network is not possible](#when-a-bridged-network-is-not-possible)
- [Initializing a slot and creating the users on the simulator](#initializing-a-slot-and-creating-the-users-on-the-simulator)
- [Using the p11tool2](#using-the-p11tool2)
- [Using the CAT tool](#using-the-cat-tool)
<!-- TOC -->
## Installing the simulator
The simulator is a 32-bit ELF application on Linux.
### ARM
When developing on 64-bit ARM system, such as a recent MacBook, the easiest way to run the simulator,
is to install a Windows VM, while performing the development on an Ubuntu VM configured to use Rosetta.
Make sure that the VM network is in bridge mode so that it gets an IP address from the same network as the host.
If bridging is not possible, start the VMs in NAT mode and
check [this paragraph](#when-a-bridged-network-is-not-possible)
### AMD64
When developing on a 64-bit AMD64 system, you must enable the 32 bit support by adding the i386 architecture:
```bash
sudo dpkg --add-architecture i386
```
Then
```bash
sudo apt-get update
sudo apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386
```
The Linux PKCS#11 library is a 64-bit ELF.
## Download and run the simulator
Download the simulator from the Utimaco website:
<https://support.hsm.utimaco.com/documents/20182/1924884/SecurityServerEvaluation-6.0.0.0.tar>
To run the simulator on Windows, open a terminal and run the following command:
```bash
cd u.trust_anchor_integration_eval_bundle-6.0.0.0\Software\Windows\Simulator\sim5_windows\bin
.\bl_sim5.exe -h -o -d ..\devices\
```
The simulator should launch on the IP address of the host machine and port 3001 and print the following message:
```text
Utimaco CryptoServer Simulator HSD process started
25.01.22 22:50:49 SMOS SDK Ver. 5.7.0.0 (Nov 26 2024) started [0]
25.01.22 22:50:49 Compiler Ver. 19.0
25.01.22 22:50:49 CPU clock frequency: 24000000
25.01.22 22:50:49 Devices directory: '..\devices\'
25.01.22 22:50:51 Sensory Controller Ver. 2.0.0.42 [0/0]
25.01.22 22:50:51 Real Random Number Generator initialized with:
RESEED_INTERVAL = 1000
PREDICTION_RESISTANCE = 0
REALRANDOM_SHARE = 3
25.01.22 22:50:51 Pseudo Random Number Generator initialized with:
RESEED_INTERVAL = 1000
PREDICTION_RESISTANCE = 0
REALRANDOM_SHARE = 0
25.01.22 22:50:51 Load module 'adm.msc' from FLASHFILE
25.01.22 22:50:51 Load module 'cmds.msc' from FLASHFILE
25.01.22 22:50:51 CMDS: .pscf support enabled
25.01.22 22:50:51 Load module 'crypt.msc' from FLASHFILE
...
```
## Configure the PKCS#11 connection on the KMS server
Install a Linux VM in bridge mode and make sure it can access the simulator
```text
telnet <simulator_ip> 3001
```
Download the simulator from the Utimaco website:
<https://support.hsm.utimaco.com/documents/20182/1924884/SecurityServerEvaluation-6.0.0.0.tar>
### PKCS#11 library
Copy the PKCS#11 library in `Software/Linux/Crypto_APIs/PKCS11_R3/lib/libcs_pkcs11_R3.so` to the `/lib` directory:
### Configuration file
Create a configuration directory and copy a sample configuration file
```bash
sudo mkdir -p /etc/utimaco
sudo chmod 755 /etc/utimaco
sudo cp u.trust_anchor_integration_eval_bundle-6.0.0.0/Software/Linux/Crypto_APIs/PKCS11_R3/sample/cs_pkcs11_R3.cfg /etc/utimaco/
```
Edit the configuration file to enable logging and set the simulator IP address and port:
```bash
sudo vim /etc/utimaco/cs_pkcs11_R3.cfg
```
```text
...
Logpath = /tmp
...
Logging = 3
...
Device = 3001@<simulator_ip>
...
```
Then make the PKCS#11 configuration file available to the library and tools:
```bash
export CS_PKCS11_R3_CFG=/etc/utimaco/cs_pkcs11_R3.cfg
```
### Test the PKCS#11 configuration
```bash
cd u.trust_anchor_integration_eval_bundle-6.0.0.0/Software/Linux/Administration
./p11tool2 Slot=0 GetSlotInfo
```
The output should be similar to:
```text
CK_SLOT_INFO (slot ID: 0x00000000):
slotDescription 33303031 40313932 2e313638 2e36382e |3001@192.168.68.|
3633202d 20534c4f 545f3030 30302020 |63 - SLOT_0000 |
20202020 20202020 20202020 20202020 | |
20202020 20202020 20202020 20202020 | |
manufacturerID 5574696d 61636f20 49532047 6d624820 |Utimaco IS GmbH |
20202020 20202020 20202020 20202020 | |
flags: 0x00000005
CKF_TOKEN_PRESENT : CK_TRUE
CKF_REMOVABLE_DEVICE : CK_FALSE
CKF_HW_SLOT : CK_TRUE
hardwareVersion : 5.02
firmwareVersion : 6.00
```
## When a bridged network is not possible
Say we have 2 VMs (one Linux with the PKCS#11 library, one Windows with the simulator)
running on a macos host. The simulator is listening on port 3001 on the Windows VM.
```text
Linux VM <----> macos host <----> Windows VM
192.168.177.25
192.168.65.3 192.168.65.1
192.168.161.1 192.168.161.138
```
Use an ssh tunnel to forward the port 3001 of the Windows VM to the 3001 of the Linux VM,
via the macOS host.
On the Linux VM, run
```sh
ssh -L 3001:192.168.161.138:3001 <macos_user>@192.168.65.1 -N -f
```
(the `-N` `-f` switches run the port forwarding in the background without opening a shell)
Update the PKCS#11 configuration file to point to localhost:
```sh
sudo vim /etc/utimaco/cs_pkcs11_R3.cfg
```
Set the Device to `3001@localhost`
Then check that the simulator is now accessible on port 3001 at localhost:
```sh
./p11tool2 Slot=0 GetSlotInfo
```
## Initializing a slot and creating the users on the simulator
A token must be initialized in slot 0 before it can be used; the Security Officer (SO) and User PINs must be set.
### Using the p11tool2
Due to a bug in the simulator, the Security Officer PIN must be set **then changed** before the User PIN can be set **then changed**.
```bash
# set the SO PIN to 11223344
./p11tool2 Slot=0 login=ADMIN,./key/ADMIN_SIM.key InitToken=11223344
# Change the SO PIN to 12345678
./p11tool2 Slot=0 LoginSO=11223344 SetPin=11223344,12345678
```
Failing to change the SO PIN before setting the User PIN will result in the following error: `Error 0x000001B8 (
CKR_PIN_TOO_WEAK)`
```bash
# Set the User PIN to 11223344
./p11tool2 Slot=0 LoginSO=12345678 InitPin=11223344
# Change the User PIN to 12345678
./p11tool2 Slot=0 LoginUser=11223344 SetPin=11223344,12345678
```
Now, both the SO and User PINs have been set to 12345678.
To list objects on Slot 0, use:
```bash
./p11tool2 Slot=0 LoginUser=12345678 ListObjects
```
### Using the CAT tool
Use the CAT tool and make sure you can login as Admin using the
ADMIN_SIM.key.
The CAT tool is a java app and is available in the `Software` directory. It requires the Oracle 8 JDK to run properly.
Then, copy to that directory the the `cs_pkcs11_R3.cfg` file and launch the java `p11cat` tool.
Use this tool to initialize slot 0 amd assign a Security Officer PIN and an User PIN.
The users will appear as `SO_0000` and `USER_0000` in the cat tool.
**Change their PIN** in the CAT tool to something else, or when using them through the PKCS#11 library,
you will keep getting `CKR_PIN_TOO_WEAK` (440) errors.
The user PIN is what should be passed to the KMS.

8
crate/hsm/utimaco/src/lib.rs Normal file
View file

@ -0,0 +1,8 @@
//! Copyright 2024 Cosmian Tech SAS
#[cfg(test)]
#[cfg(feature = "utimaco")]
mod tests;
/// The Utimaco HSM is fully supported by the BaseHsm implementation
pub type Utimaco = cosmian_kms_base_hsm::BaseHsm;

140
crate/pkcs11/proteccio/src/tests.rs → crate/hsm/utimaco/src/tests.rs
View file

@ -1,62 +1,33 @@
//! These tests require a connection to a working HSM and are gated behind the `proteccio` feature.
//! These tests require a connection to a working HSM and are gated behind the `utimaco` feature.
//! To run a test, cd into the crate directory and run (replace `XXX` with the actual password):
//! ```
//! HSM_USER_PASSWORD=XXX cargo test --target x86_64-unknown-linux-gnu --features proteccio -- tests::test_all
//! HSM_USER_PASSWORD=XXX cargo test --target x86_64-unknown-linux-gnu --features utimaco -- tests::test_all
//! ```
use std::{
collections::HashMap,
ptr,
sync::{Arc, Once},
thread,
};
use std::{collections::HashMap, ptr, sync::Arc, thread};
use cosmian_kms_base_hsm::{
test_helpers::{get_hsm_password, initialize_logging},
AesKeySize, HError, HResult, HsmEncryptionAlgorithm, RsaKeySize, SlotManager,
};
use cosmian_kms_interfaces::{HsmObjectFilter, KeyMaterial, KeyType};
use libloading::Library;
use pkcs11_sys::{CKF_OS_LOCKING_OK, CKR_OK, CK_C_INITIALIZE_ARGS, CK_RV, CK_VOID_PTR};
use tracing::{info, Level};
use tracing_subscriber::FmtSubscriber;
use tracing::info;
use uuid::Uuid;
use crate::{
AesKeySize, PError, PResult, Proteccio, ProteccioEncryptionAlgorithm, RsaKeySize, SlotManager,
};
use crate::Utimaco;
static TRACING_INIT: Once = Once::new();
fn initialize_logging() {
TRACING_INIT.call_once(|| {
let subscriber = FmtSubscriber::builder()
.with_max_level(Level::INFO) // Adjust the level as needed
.with_writer(std::io::stdout)
.finish();
tracing::subscriber::set_global_default(subscriber)
.expect("Setting default subscriber failed");
});
}
fn get_hsm_password() -> PResult<String> {
let user_password = option_env!("HSM_USER_PASSWORD")
.ok_or_else(|| {
PError::Default(
"The user password for the HSM is not set. Please set the HSM_USER_PASSWORD \
environment variable"
.to_string(),
)
})?
.to_string();
Ok(user_password)
}
fn get_slot() -> PResult<Arc<SlotManager>> {
fn get_slot() -> HResult<Arc<SlotManager>> {
let user_password = get_hsm_password()?;
let passwords = HashMap::from([(0x04, Some(user_password.clone()))]);
let hsm = Proteccio::instantiate("/lib/libnethsm64.so", passwords)?;
let manager = hsm.get_slot(0x04)?;
let passwords = HashMap::from([(0x00, Some(user_password.clone()))]);
let hsm = Utimaco::instantiate("/lib/libcs_pkcs11_R3.so", passwords)?;
let manager = hsm.get_slot(0x00)?;
Ok(manager)
}
#[test]
fn test_all() -> PResult<()> {
fn test_all() -> HResult<()> {
test_hsm_get_info()?;
test_destroy_all()?;
test_generate_aes_key()?;
@ -73,12 +44,12 @@ fn test_all() -> PResult<()> {
}
#[test]
fn low_level_test() -> PResult<()> {
let path = "/lib/libnethsm64.so";
fn low_level_test() -> HResult<()> {
let path = "/lib/libcs_pkcs11_R3.so";
let library = unsafe { Library::new(path) }?;
let init = unsafe { library.get::<fn(pInitArgs: CK_VOID_PTR) -> CK_RV>(b"C_Initialize") }?;
let init = unsafe { library.get::<fn(p_init_args: CK_VOID_PTR) -> CK_RV>(b"C_Initialize") }?;
let mut pInitArgs = CK_C_INITIALIZE_ARGS {
let mut p_init_args = CK_C_INITIALIZE_ARGS {
CreateMutex: None,
DestroyMutex: None,
LockMutex: None,
@ -86,23 +57,23 @@ fn low_level_test() -> PResult<()> {
flags: CKF_OS_LOCKING_OK,
pReserved: ptr::null_mut(),
};
let rv = init(&mut pInitArgs as *const CK_C_INITIALIZE_ARGS as CK_VOID_PTR);
let rv = init(&mut p_init_args as *const CK_C_INITIALIZE_ARGS as CK_VOID_PTR);
assert_eq!(rv, CKR_OK);
Ok(())
}
#[test]
fn test_hsm_get_info() -> PResult<()> {
fn test_hsm_get_info() -> HResult<()> {
initialize_logging();
let hsm = Proteccio::instantiate("/lib/libnethsm64.so", HashMap::new())?;
let hsm = Utimaco::instantiate("/lib/libcs_pkcs11_R3.so", HashMap::new())?;
let info = hsm.get_info()?;
info!("Connected to the HSM: {info}");
Ok(())
}
#[test]
fn test_generate_aes_key() -> PResult<()> {
fn test_generate_aes_key() -> HResult<()> {
initialize_logging();
let key_id = Uuid::new_v4().to_string();
let slot = get_slot()?;
@ -144,7 +115,7 @@ fn test_generate_aes_key() -> PResult<()> {
}
#[test]
fn test_generate_rsa_keypair() -> PResult<()> {
fn test_generate_rsa_keypair() -> HResult<()> {
initialize_logging();
let sk_id = Uuid::new_v4().to_string();
let pk_id = sk_id.clone() + "_pk ";
@ -205,13 +176,12 @@ fn test_generate_rsa_keypair() -> PResult<()> {
}
#[test]
fn test_rsa_key_wrap() -> PResult<()> {
fn test_rsa_key_wrap() -> HResult<()> {
initialize_logging();
let key_id = Uuid::new_v4().to_string();
let slot = get_slot()?;
let session = slot.open_session(true)?;
let symmetric_key = session.generate_aes_key(key_id.as_bytes(), AesKeySize::Aes256, true)?;
info!("Symmetric key handle: {symmetric_key}");
let sk_id = Uuid::new_v4().to_string();
let pk_id = sk_id.clone() + "_pk ";
let (sk, pk) = session.generate_rsa_key_pair(
@ -220,17 +190,16 @@ fn test_rsa_key_wrap() -> PResult<()> {
RsaKeySize::Rsa2048,
true,
)?;
info!("RSA handles sk: {sk}, pl: {pk}");
let encrypted_key = session.wrap_aes_key_with_rsa_oaep(pk, symmetric_key)?;
assert_eq!(encrypted_key.len(), 2048 / 8);
let decrypted_key =
session.unwrap_aes_key_with_rsa_oaep(sk, &encrypted_key, "another_label")?;
info!("Unwrapped symmetric key handle: {}", decrypted_key);
info!("Unwrapped symmetric key with handle: {}", decrypted_key);
Ok(())
}
#[test]
fn test_rsa_pkcs_encrypt() -> PResult<()> {
fn test_rsa_pkcs_encrypt() -> HResult<()> {
initialize_logging();
let slot = get_slot()?;
let session = slot.open_session(true)?;
@ -243,20 +212,16 @@ fn test_rsa_pkcs_encrypt() -> PResult<()> {
RsaKeySize::Rsa2048,
true,
)?;
info!("RSA handles sk: {sk}, pl: {pk}");
let enc = session.encrypt(pk, ProteccioEncryptionAlgorithm::RsaPkcsV15, data)?;
let enc = session.encrypt(pk, HsmEncryptionAlgorithm::RsaPkcsV15, data)?;
assert_eq!(enc.ciphertext.len(), 2048 / 8);
let plaintext = session.decrypt(
sk,
ProteccioEncryptionAlgorithm::RsaPkcsV15,
&enc.ciphertext,
)?;
let plaintext = session.decrypt(sk, HsmEncryptionAlgorithm::RsaPkcsV15, &enc.ciphertext)?;
assert_eq!(plaintext.as_slice(), data);
info!("Successfully encrypted/decrypted with RSA PKCS");
Ok(())
}
#[test]
fn test_rsa_oaep_encrypt() -> PResult<()> {
fn test_rsa_oaep_encrypt() -> HResult<()> {
initialize_logging();
let slot = get_slot()?;
let session = slot.open_session(true)?;
@ -269,16 +234,16 @@ fn test_rsa_oaep_encrypt() -> PResult<()> {
RsaKeySize::Rsa2048,
true,
)?;
info!("RSA handles sk: {sk}, pl: {pk}");
let enc = session.encrypt(pk, ProteccioEncryptionAlgorithm::RsaOaep, data)?;
let enc = session.encrypt(pk, HsmEncryptionAlgorithm::RsaOaep, data)?;
assert_eq!(enc.ciphertext.len(), 2048 / 8);
let plaintext = session.decrypt(sk, ProteccioEncryptionAlgorithm::RsaOaep, &enc.ciphertext)?;
let plaintext = session.decrypt(sk, HsmEncryptionAlgorithm::RsaOaep, &enc.ciphertext)?;
assert_eq!(plaintext.as_slice(), data);
info!("Successfully encrypted/decrypted with RSA OAEP");
Ok(())
}
#[test]
fn test_aes_gcm_encrypt() -> PResult<()> {
fn test_aes_gcm_encrypt() -> HResult<()> {
initialize_logging();
let slot = get_slot()?;
let session = slot.open_session(true)?;
@ -286,13 +251,13 @@ fn test_aes_gcm_encrypt() -> PResult<()> {
let key_id = Uuid::new_v4().to_string();
let sk = session.generate_aes_key(key_id.as_bytes(), AesKeySize::Aes256, true)?;
info!("AES key handle: {sk}");
let enc = session.encrypt(sk, ProteccioEncryptionAlgorithm::AesGcm, data)?;
let enc = session.encrypt(sk, HsmEncryptionAlgorithm::AesGcm, data)?;
assert_eq!(enc.ciphertext.len(), data.len());
assert_eq!(enc.tag.clone().unwrap_or_default().len(), 16);
assert_eq!(enc.iv.clone().unwrap_or_default().len(), 12);
let plaintext = session.decrypt(
sk,
ProteccioEncryptionAlgorithm::AesGcm,
HsmEncryptionAlgorithm::AesGcm,
[
enc.iv.unwrap_or_default(),
enc.ciphertext,
@ -302,11 +267,12 @@ fn test_aes_gcm_encrypt() -> PResult<()> {
.as_slice(),
)?;
assert_eq!(plaintext.as_slice(), data);
info!("Successfully encrypted/decrypted with AES GCM");
Ok(())
}
#[test]
fn multi_threaded_rsa_encrypt_decrypt_test() -> PResult<()> {
fn multi_threaded_rsa_encrypt_decrypt_test() -> HResult<()> {
initialize_logging();
// Initialize the HSM once and share it across threads
@ -327,16 +293,15 @@ fn multi_threaded_rsa_encrypt_decrypt_test() -> PResult<()> {
true,
)?;
info!("RSA handles sk: {sk}, pk: {pk}");
let encrypted_content =
session.encrypt(pk, ProteccioEncryptionAlgorithm::RsaOaep, data)?;
let encrypted_content = session.encrypt(pk, HsmEncryptionAlgorithm::RsaOaep, data)?;
assert_eq!(encrypted_content.ciphertext.len(), 2048 / 8);
let plaintext = session.decrypt(
sk,
ProteccioEncryptionAlgorithm::RsaOaep,
HsmEncryptionAlgorithm::RsaOaep,
&encrypted_content.ciphertext,
)?;
assert_eq!(plaintext.as_slice(), data);
Ok::<(), PError>(())
Ok::<(), HError>(())
});
handles.push(handle);
}
@ -344,12 +309,12 @@ fn multi_threaded_rsa_encrypt_decrypt_test() -> PResult<()> {
for handle in handles {
handle.join().expect("Thread panicked")?;
}
info!("Successfully encrypted/decrypted with RSA OAEP in multiple threads");
Ok(())
}
#[test]
fn test_list_objects() -> PResult<()> {
fn test_list_objects() -> HResult<()> {
initialize_logging();
let slot = get_slot()?;
let session = slot.open_session(true)?;
@ -403,11 +368,12 @@ fn test_list_objects() -> PResult<()> {
assert_eq!(objects.len(), 5);
let objects = session.list_objects(HsmObjectFilter::AesKey)?;
assert_eq!(objects.len(), 1);
info!("Listed all objects");
Ok(())
}
#[test]
fn test_get_key_metadata() -> PResult<()> {
fn test_get_key_metadata() -> HResult<()> {
initialize_logging();
let slot = get_slot()?;
let session = slot.open_session(true)?;
@ -418,12 +384,12 @@ fn test_get_key_metadata() -> PResult<()> {
// get the key basics
let key_type = session
.get_key_type(key_handle)?
.ok_or_else(|| PError::Default("Key not found".to_string()))?;
.ok_or_else(|| HError::Default("Key not found".to_string()))?;
assert_eq!(key_type, KeyType::AesKey);
// get the metadata
let metadata = session
.get_key_metadata(key_handle)?
.ok_or_else(|| PError::Default("Key not found".to_string()))?;
.ok_or_else(|| HError::Default("Key not found".to_string()))?;
assert_eq!(metadata.key_type, KeyType::AesKey);
assert!(metadata.sensitive);
assert_eq!(metadata.key_length_in_bits, 256);
@ -442,13 +408,13 @@ fn test_get_key_metadata() -> PResult<()> {
// get the private key basics
let key_type = session
.get_key_type(sk)?
.ok_or_else(|| PError::Default("Key not found".to_string()))?;
.ok_or_else(|| HError::Default("Key not found".to_string()))?;
assert_eq!(key_type, KeyType::RsaPrivateKey);
// get the private key metadata
let metadata = session
.get_key_metadata(sk)?
.ok_or_else(|| PError::Default("Key not found".to_string()))?;
.ok_or_else(|| HError::Default("Key not found".to_string()))?;
assert_eq!(metadata.key_type, KeyType::RsaPrivateKey);
assert_eq!(metadata.key_length_in_bits, 2048);
assert_eq!(metadata.id.as_str(), sk_id.as_str());
@ -457,22 +423,23 @@ fn test_get_key_metadata() -> PResult<()> {
// get the public key basics
let key_type = session
.get_key_type(pk)?
.ok_or_else(|| PError::Default("Key not found".to_string()))?;
.ok_or_else(|| HError::Default("Key not found".to_string()))?;
assert_eq!(key_type, KeyType::RsaPublicKey);
// get the public key metadata
let metadata = session
.get_key_metadata(pk)?
.ok_or_else(|| PError::Default("Key not found".to_string()))?;
.ok_or_else(|| HError::Default("Key not found".to_string()))?;
assert_eq!(metadata.key_type, KeyType::RsaPublicKey);
// assert!(metadata.sensitive);
assert_eq!(metadata.key_length_in_bits, 2048);
assert_eq!(metadata.id.as_str(), pk_id.as_str());
info!("Got key metadata");
Ok(())
}
#[test]
fn test_destroy_all() -> PResult<()> {
fn test_destroy_all() -> HResult<()> {
initialize_logging();
let slot = get_slot()?;
let session = slot.open_session(true)?;
@ -482,5 +449,6 @@ fn test_destroy_all() -> PResult<()> {
}
let objects = session.list_objects(HsmObjectFilter::Any)?;
assert_eq!(objects.len(), 0);
info!("Destroyed all objects");
Ok(())
}

2
crate/kmip/src/bytes_ser_de.rs
View file

@ -303,7 +303,7 @@ mod tests {
#[test]
fn test_to_leb128_len() -> Result<(), KmipError> {
let mut rng = rand::thread_rng();
let mut rng = rand::rng();
let mut ser = Serializer::new();
for i in 1..1000 {
let n = rng.next_u32();

18
crate/pkcs11/README.md
View file

@ -1,7 +1,7 @@
This directory provides
- base Rust PKCS#11 bindings and traits that ccan be used to create a PKCS#11 client or a PPKCS#11 provider
- a PKCS#11 library to interface the KMS (the `provider` crate) from a PKCS#11 compliant application such as LUKS
- a PKCS#11 wrapper to connect to an HSM (the `proteccio` crate)
[PKCS##11 documentation](https://www.cryptsoft.com/pkcs11doc/STANDARD/pkcs-11.pdf)
@ -19,19 +19,3 @@ This directory provides
The provider crate is a PKCS#11 library that interfaces the KMS. It provides a PKCS#11 library that can be used by
applications such as LUKS to interface the KMS. The `provider` crate is built from the `module` crate.
4. `hsm` crate
The `hsm` crate provides traits that should be implemented by wrapper loading HSM PKC#11 libraries.
5. `proteccio` crate
The `proteccio` crate is a PKCS#11 wrapper that connects to the Proteccio HSM. It wraps the Proteccio HSM PKCS#11
library and provides implementation of the `hsm` crate traits used by the KMS.
The PKCS#11 library is built from the `provider` crate.
The `module` crate is a modified fork of Google native_pkcs11 crate. See its readme for details.
The `sys`crate is a direct clone of the crate with the same name from the `native_pkcs11` crate. Its license is Apache
2.0.

78
crate/pkcs11/proteccio/src/lib.rs
View file

@ -1,78 +0,0 @@
//! Copyright 2024 Cosmian Tech SAS
#![allow(non_snake_case)]
#![allow(clippy::missing_safety_doc)]
extern crate core;
mod error;
pub use error::{PError, PResult};
pub use proteccio::Proteccio;
use rand::{rngs::OsRng, TryRngCore};
pub use session::{AesKeySize, ProteccioEncryptionAlgorithm, RsaKeySize, Session};
pub use slots::{ObjectHandlesCache, SlotManager};
mod proteccio;
mod session;
mod kms_hsm;
mod slots;
#[cfg(test)]
#[cfg(feature = "proteccio")]
mod tests;
/// This is a macro because of the mut pointer to the params
/// We therefore want this code to be inlined
#[macro_export]
macro_rules! aes_mechanism {
($nonce:expr) => {{
let mut params = CK_AES_GCM_PARAMS {
pIv: $nonce as *mut u8,
ulIvLen: 12,
ulIvBits: 96,
pAAD: std::ptr::null_mut(),
ulAADLen: 0,
ulTagBits: 128,
};
CK_MECHANISM {
mechanism: CKM_AES_GCM,
pParameter: &mut params as *mut _ as CK_VOID_PTR,
ulParameterLen: std::mem::size_of::<CK_AES_GCM_PARAMS>() as CK_ULONG,
}
}};
}
#[macro_export]
macro_rules! rsa_mechanism {
($algorithm:expr) => {
match $algorithm {
ProteccioEncryptionAlgorithm::RsaPkcsV15 => CK_MECHANISM {
mechanism: CKM_RSA_PKCS,
pParameter: std::ptr::null_mut(),
ulParameterLen: 0,
},
ProteccioEncryptionAlgorithm::RsaOaep => {
let mut params = CK_RSA_PKCS_OAEP_PARAMS {
hashAlg: CKM_SHA256,
mgf: CKG_MGF1_SHA256,
source: CKZ_DATA_SPECIFIED,
pSourceData: std::ptr::null_mut(),
ulSourceDataLen: 0,
};
CK_MECHANISM {
mechanism: CKM_RSA_PKCS_OAEP,
pParameter: &mut params as *mut _ as CK_VOID_PTR,
ulParameterLen: std::mem::size_of::<CK_RSA_PKCS_OAEP_PARAMS>() as CK_ULONG,
}
}
_ => return Err(PError::Default("expecting an RSA algorithm".to_string())),
}
};
}
fn generate_random_nonce<const T: usize>() -> PResult<[u8; T]> {
let mut bytes = [0u8; T];
OsRng
.try_fill_bytes(&mut bytes)
.map_err(|e| PError::Default(format!("Error generating random nonce: {}", e)))?;
Ok(bytes)
}

258
crate/pkcs11/proteccio/src/proteccio.rs
View file

@ -1,258 +0,0 @@
use std::{
collections::HashMap,
ffi::CStr,
fmt,
fmt::{Display, Formatter},
ptr,
sync::{Arc, Mutex},
};
use libloading::Library;
use pkcs11_sys::*;
use crate::{PError, PResult, SlotManager};
struct SlotState {
password: Option<String>,
slot: Option<Arc<SlotManager>>,
}
pub struct Proteccio {
hsm_lib: Arc<HsmLib>,
slots: Mutex<HashMap<usize, SlotState>>,
}
impl Proteccio {
pub fn instantiate<P>(path: P, passwords: HashMap<usize, Option<String>>) -> PResult<Self>
where
P: AsRef<std::ffi::OsStr>,
{
let hsm_lib = Arc::new(HsmLib::instantiate(path)?);
let mut slots = HashMap::with_capacity(passwords.len());
for (k, v) in passwords.iter() {
slots.insert(
*k,
SlotState {
password: v.clone(),
slot: None,
},
);
}
Ok(Proteccio {
hsm_lib,
slots: Mutex::new(slots),
})
}
/// Get a slot
/// If a slot has already been opened, returns the opened slot.
/// To close a slot before re-opening it with another password, call `close_slot()` first
pub fn get_slot(&self, slot_id: usize) -> PResult<Arc<SlotManager>> {
let mut slots = self.slots.lock().expect("failed to lock slots");
// check if we are supposed to use that slot
if let Some(slot_state) = slots.get_mut(&slot_id) {
if let Some(s) = &slot_state.slot {
Ok(s.clone())
} else {
// instantiate a new slot
let manager = Arc::new(SlotManager::instantiate(
self.hsm_lib.clone(),
slot_id,
slot_state.password.clone(),
)?);
slot_state.slot = Some(manager.clone());
Ok(manager)
}
} else {
Err(PError::Default(format!("slot {slot_id} is not accessible")))
}
}
pub fn close_slot(&self, slot_id: usize) -> PResult<()> {
let mut slots = self.slots.lock().expect("failed to lock slots");
slots.remove(&slot_id);
Ok(())
}
pub fn get_info(&self) -> PResult<Info> {
unsafe {
let mut info = CK_INFO::default();
let rv =
self.hsm_lib.C_GetInfo.ok_or_else(|| {
PError::Default("C_GetInfo not available on library".to_string())
})?(&mut info);
if rv != CKR_OK {
return Err(PError::Default("Failed getting HSM info".to_string()));
}
Ok(info.into())
}
}
}
#[allow(dead_code)]
pub struct HsmLib {
_library: Library,
pub(crate) C_Initialize: CK_C_Initialize,
pub(crate) C_Finalize: CK_C_Finalize,
pub(crate) C_OpenSession: CK_C_OpenSession,
pub(crate) C_CloseSession: CK_C_CloseSession,
pub(crate) C_DestroyObject: CK_C_DestroyObject,
pub(crate) C_Decrypt: CK_C_Decrypt,
pub(crate) C_DecryptInit: CK_C_DecryptInit,
pub(crate) C_DecryptUpdate: CK_C_DecryptUpdate,
pub(crate) C_DecryptFinal: CK_C_DecryptFinal,
pub(crate) C_Encrypt: CK_C_Encrypt,
pub(crate) C_EncryptInit: CK_C_EncryptInit,
pub(crate) C_EncryptUpdate: CK_C_EncryptUpdate,
pub(crate) C_EncryptFinal: CK_C_EncryptFinal,
pub(crate) C_FindObjectsInit: CK_C_FindObjectsInit,
pub(crate) C_FindObjects: CK_C_FindObjects,
pub(crate) C_FindObjectsFinal: CK_C_FindObjectsFinal,
pub(crate) C_GenerateKey: CK_C_GenerateKey,
pub(crate) C_GenerateKeyPair: CK_C_GenerateKeyPair,
pub(crate) C_GenerateRandom: CK_C_GenerateRandom,
pub(crate) C_GetAttributeValue: CK_C_GetAttributeValue,
pub(crate) C_GetInfo: CK_C_GetInfo,
pub(crate) C_Login: CK_C_Login,
pub(crate) C_Logout: CK_C_Logout,
pub(crate) C_WrapKey: CK_C_WrapKey,
pub(crate) C_UnwrapKey: CK_C_UnwrapKey,
}
impl HsmLib {
fn instantiate<P>(path: P) -> PResult<Self>
where
P: AsRef<std::ffi::OsStr>,
{
unsafe {
let library = Library::new(path)?;
let hsm_lib = HsmLib {
C_Initialize: Some(*library.get(b"C_Initialize")?),
C_Finalize: Some(*library.get(b"C_Finalize")?),
C_OpenSession: Some(*library.get(b"C_OpenSession")?),
C_CloseSession: Some(*library.get(b"C_CloseSession")?),
C_Encrypt: Some(*library.get(b"C_Encrypt")?),
C_EncryptInit: Some(*library.get(b"C_EncryptInit")?),
C_EncryptUpdate: Some(*library.get(b"C_EncryptUpdate")?),
C_EncryptFinal: Some(*library.get(b"C_EncryptFinal")?),
C_Decrypt: Some(*library.get(b"C_Decrypt")?),
C_DecryptInit: Some(*library.get(b"C_DecryptInit")?),
C_DecryptUpdate: Some(*library.get(b"C_DecryptUpdate")?),
C_DecryptFinal: Some(*library.get(b"C_DecryptFinal")?),
C_DestroyObject: Some(*library.get(b"C_DestroyObject")?),
C_FindObjectsInit: Some(*library.get(b"C_FindObjectsInit")?),
C_FindObjects: Some(*library.get(b"C_FindObjects")?),
C_FindObjectsFinal: Some(*library.get(b"C_FindObjectsFinal")?),
C_GenerateKey: Some(*library.get(b"C_GenerateKey")?),
C_GenerateKeyPair: Some(*library.get(b"C_GenerateKeyPair")?),
C_GenerateRandom: Some(*library.get(b"C_GenerateRandom")?),
C_GetAttributeValue: Some(*library.get(b"C_GetAttributeValue")?),
C_GetInfo: Some(*library.get(b"C_GetInfo")?),
C_Login: Some(*library.get(b"C_Login")?),
C_Logout: Some(*library.get(b"C_Logout")?),
C_WrapKey: Some(*library.get(b"C_WrapKey")?),
C_UnwrapKey: Some(*library.get(b"C_UnwrapKey")?),
// we need to keep the library alive
_library: library,
};
Self::initialize(&hsm_lib)?;
Ok(hsm_lib)
}
}
fn initialize(hsm_lib: &HsmLib) -> PResult<()> {
let pInitArgs = CK_C_INITIALIZE_ARGS {
CreateMutex: None,
DestroyMutex: None,
LockMutex: None,
UnlockMutex: None,
flags: CKF_OS_LOCKING_OK,
pReserved: ptr::null_mut(),
};
unsafe {
// let rv = self.hsm.C_Initialize.deref()(&pInitArgs);
let rv = hsm_lib.C_Initialize.ok_or_else(|| {
PError::Default("C_Initialize not available on library".to_string())
})?(&pInitArgs as *const CK_C_INITIALIZE_ARGS as CK_VOID_PTR);
if rv != CKR_OK {
return Err(PError::Default("Failed initializing the HSM".to_string()));
}
Ok(())
}
}
fn finalize(&self) -> PResult<()> {
unsafe {
let rv = self.C_Finalize.ok_or_else(|| {
PError::Default("C_Finalize not available on library".to_string())
})?(ptr::null_mut());
if rv != CKR_OK {
return Err(PError::Default("Failed to finalize the HSM".to_string()));
}
Ok(())
}
}
}
impl Drop for HsmLib {
fn drop(&mut self) {
let _ = self.finalize();
}
}
pub struct Info {
pub cryptokiVersion: (u8, u8),
pub manufacturerID: String,
pub flags: u64,
pub libraryDescription: String,
pub libraryVersion: (u8, u8),
}
impl From<CK_INFO> for Info {
fn from(info: CK_INFO) -> Self {
#[cfg(target_os = "windows")]
let flags = u64::from(info.flags);
#[cfg(not(target_os = "windows"))]
let flags = info.flags;
Info {
cryptokiVersion: (info.cryptokiVersion.major, info.cryptokiVersion.minor),
manufacturerID: CStr::from_bytes_until_nul(&info.manufacturerID)
.unwrap_or_default()
.to_string_lossy()
.to_string(),
flags,
libraryDescription: CStr::from_bytes_until_nul(&info.libraryDescription)
.unwrap_or_default()
.to_string_lossy()
.to_string(),
libraryVersion: (info.libraryVersion.major, info.libraryVersion.minor),
}
}
}
impl Display for Info {
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
write!(
f,
"Cryptoki Version: {}.{}\nManufacturer ID: {}\nFlags: {}\nLibrary Description: \
{}\nLibrary Version: {}.{}",
self.cryptokiVersion.0,
self.cryptokiVersion.1,
self.manufacturerID,
self.flags,
self.libraryDescription,
self.libraryVersion.0,
self.libraryVersion.1
)
}
}

127
crate/pkcs11/proteccio/src/session/aes.rs
View file

@ -1,127 +0,0 @@
use std::ptr;
use pkcs11_sys::{
CKA_CLASS, CKA_DECRYPT, CKA_ENCRYPT, CKA_EXTRACTABLE, CKA_KEY_TYPE, CKA_LABEL, CKA_PRIVATE,
CKA_SENSITIVE, CKA_TOKEN, CKA_VALUE_LEN, CKK_AES, CKM_AES_KEY_GEN, CKO_SECRET_KEY, CKR_OK,
CK_ATTRIBUTE, CK_ATTRIBUTE_PTR, CK_BBOOL, CK_FALSE, CK_MECHANISM, CK_MECHANISM_PTR,
CK_OBJECT_HANDLE, CK_TRUE, CK_ULONG, CK_VOID_PTR,
};
use crate::{session::Session, PError, PResult};
pub enum AesKeySize {
Aes128,
Aes256,
}
/// AES key template
/// If sensitive is true, the key is not exportable
/// Proteccio does not allow setting the ID attribute for secret keys so we use the LABEL
pub(crate) const fn aes_key_template(
id: &[u8],
size: CK_ULONG,
sensitive: bool,
) -> [CK_ATTRIBUTE; 10] {
let sensitive: CK_BBOOL = if sensitive { CK_TRUE } else { CK_FALSE };
[
CK_ATTRIBUTE {
type_: CKA_CLASS,
pValue: &CKO_SECRET_KEY as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_ULONG>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_KEY_TYPE,
pValue: &CKK_AES as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_ULONG>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_TOKEN,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_BBOOL>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_VALUE_LEN,
pValue: &size as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_ULONG>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_ENCRYPT,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_BBOOL>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_DECRYPT,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_BBOOL>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_LABEL,
pValue: id.as_ptr() as CK_VOID_PTR,
ulValueLen: id.len() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_PRIVATE,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_BBOOL>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_SENSITIVE,
pValue: &sensitive as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_BBOOL>() as CK_ULONG,
},
CK_ATTRIBUTE {
type_: CKA_EXTRACTABLE,
pValue: &CK_TRUE as *const _ as CK_VOID_PTR,
ulValueLen: size_of::<CK_BBOOL>() as CK_ULONG,
},
]
}
impl Session {
/// Generate an AES key
///
/// If exportable is set to `false`, the `sensitive` flag is set to true,
/// and the key will not be exportable.
pub fn generate_aes_key(
&self,
id: &[u8],
size: AesKeySize,
sensitive: bool,
) -> PResult<CK_OBJECT_HANDLE> {
unsafe {
let ck_fn = self.hsm().C_GenerateKey.ok_or_else(|| {
PError::Default("C_GenerateKey not available on library".to_string())
})?;
let size = match size {
AesKeySize::Aes128 => 16,
AesKeySize::Aes256 => 32,
} as CK_ULONG;
let mut mechanism = CK_MECHANISM {
mechanism: CKM_AES_KEY_GEN,
pParameter: ptr::null_mut(),
ulParameterLen: 0,
};
let mut template = aes_key_template(id, size, sensitive);
let pMechanism: CK_MECHANISM_PTR = &mut mechanism;
let pMutTemplate: CK_ATTRIBUTE_PTR = template.as_mut_ptr();
let mut aes_key_handle = CK_OBJECT_HANDLE::default();
#[cfg(target_os = "windows")]
let len = u32::try_from(template.len())?;
#[cfg(not(target_os = "windows"))]
let len = u64::try_from(template.len())?;
let rv = ck_fn(
self.session_handle(),
pMechanism,
pMutTemplate,
len,
&mut aes_key_handle,
);
if rv != CKR_OK {
return Err(PError::Default("Failed generating key".to_string()));
}
self.object_handles_cache()
.insert(id.to_vec(), aes_key_handle);
Ok(aes_key_handle)
}
}
}

5
crate/pkcs11/proteccio/src/session/mod.rs
View file

@ -1,5 +0,0 @@
mod aes;
mod rsa;
mod session_impl;
pub use session_impl::{AesKeySize, ProteccioEncryptionAlgorithm, RsaKeySize, Session};

150
crate/pkcs11/proteccio/src/slots.rs
View file

@ -1,150 +0,0 @@
use std::{
num::NonZeroUsize,
ptr,
sync::{Arc, Mutex},
};
use lru::LruCache;
use pkcs11_sys::{
CKF_RW_SESSION, CKF_SERIAL_SESSION, CKR_OK, CKR_USER_ALREADY_LOGGED_IN, CKU_USER, CK_FLAGS,
CK_OBJECT_HANDLE, CK_SESSION_HANDLE, CK_SLOT_ID, CK_ULONG, CK_UTF8CHAR_PTR,
};
use tracing::warn;
use crate::{proteccio::HsmLib, PError, PResult, Session};
pub struct ObjectHandlesCache(Mutex<LruCache<Vec<u8>, CK_OBJECT_HANDLE>>);
impl Default for ObjectHandlesCache {
fn default() -> Self {
Self::new()
}
}
impl ObjectHandlesCache {
pub fn new() -> Self {
#[allow(unsafe_code)]
let max = unsafe { NonZeroUsize::new_unchecked(100) };
ObjectHandlesCache(Mutex::new(LruCache::new(max)))
}
pub fn get(&self, key: &[u8]) -> Option<CK_OBJECT_HANDLE> {
self.0
.lock()
.expect("Proteccio: failed to lock the handles cache")
.get(key)
.copied()
}
pub fn insert(&self, key: Vec<u8>, value: CK_OBJECT_HANDLE) {
self.0
.lock()
.expect("Proteccio: failed to lock the handles cache")
.put(key, value);
}
pub fn remove(&self, key: &[u8]) {
self.0
.lock()
.expect("Proteccio: failed to lock the handles cache")
.pop(key);
}
}
pub struct SlotManager {
hsm_lib: Arc<HsmLib>,
slot_id: usize,
object_handles_cache: Arc<ObjectHandlesCache>,
_login_session: Option<Session>,
}
impl SlotManager {
pub fn instantiate(
hsm_lib: Arc<HsmLib>,
slot_id: usize,
login_password: Option<String>,
) -> PResult<Self> {
let object_handles_cache = Arc::new(ObjectHandlesCache::new());
if let Some(password) = login_password {
let login_session = Self::open_session_(
&hsm_lib,
slot_id,
false,
object_handles_cache.clone(),
Some(password),
)?;
Ok(SlotManager {
hsm_lib,
slot_id,
object_handles_cache,
_login_session: Some(login_session),
})
} else {
Ok(SlotManager {
hsm_lib,
slot_id,
object_handles_cache,
_login_session: None,
})
}
}
pub fn open_session(&self, read_write: bool) -> PResult<Session> {
Self::open_session_(
&self.hsm_lib,
self.slot_id,
read_write,
self.object_handles_cache.clone(),
None,
)
}
fn open_session_(
hsm_lib: &Arc<HsmLib>,
slot_id: usize,
read_write: bool,
object_handles_cache: Arc<ObjectHandlesCache>,
login_password: Option<String>,
) -> PResult<Session> {
let slot_id: CK_SLOT_ID = slot_id as CK_SLOT_ID;
let flags: CK_FLAGS = if read_write {
CKF_RW_SESSION | CKF_SERIAL_SESSION
} else {
CKF_SERIAL_SESSION
};
let mut session_handle: CK_SESSION_HANDLE = 0;
unsafe {
let rv = hsm_lib.C_OpenSession.ok_or_else(|| {
PError::Default("C_OpenSession not available on library".to_string())
})?(slot_id, flags, ptr::null_mut(), None, &mut session_handle);
if rv != CKR_OK {
return Err(PError::Default(format!(
"Proteccio: Failed opening a session: {rv}å"
)));
}
if let Some(password) = login_password.as_ref() {
let mut pwd_bytes = password.as_bytes().to_vec();
let rv = hsm_lib.C_Login.ok_or_else(|| {
PError::Default("C_Login not available on library".to_string())
})?(
session_handle,
CKU_USER,
pwd_bytes.as_mut_ptr() as CK_UTF8CHAR_PTR,
pwd_bytes.len() as CK_ULONG,
);
if rv == CKR_USER_ALREADY_LOGGED_IN {
warn!("user already logged in, ignoring logging");
} else if rv != CKR_OK {
return Err(PError::Default("Failed logging in".to_string()));
}
}
Ok(Session::new(
hsm_lib.clone(),
session_handle,
object_handles_cache,
login_password.is_some(),
))
}
}
}

242
crate/pkcs11/sys/src/lib.rs
View file

@ -29,3 +29,245 @@ pub const CK_UNAVAILABLE_INFORMATION: u32 = std::u32::MAX;
mod pkcs11_unix;
#[cfg(not(target_os = "windows"))]
pub use pkcs11_unix::*;
pub fn get_ck_rv_name(value: CK_RV) -> &'static str {
match value {
0 => "CKR_OK",
1 => "CKR_CANCEL",
2 => "CKR_HOST_MEMORY",
3 => "CKR_SLOT_ID_INVALID",
5 => "CKR_GENERAL_ERROR",
6 => "CKR_FUNCTION_FAILED",
7 => "CKR_ARGUMENTS_BAD",
8 => "CKR_NO_EVENT",
9 => "CKR_NEED_TO_CREATE_THREADS",
10 => "CKR_CANT_LOCK",
16 => "CKR_ATTRIBUTE_READ_ONLY",
17 => "CKR_ATTRIBUTE_SENSITIVE",
18 => "CKR_ATTRIBUTE_TYPE_INVALID",
19 => "CKR_ATTRIBUTE_VALUE_INVALID",
27 => "CKR_ACTION_PROHIBITED",
32 => "CKR_DATA_INVALID",
33 => "CKR_DATA_LEN_RANGE",
48 => "CKR_DEVICE_ERROR",
49 => "CKR_DEVICE_MEMORY",
50 => "CKR_DEVICE_REMOVED",
64 => "CKR_ENCRYPTED_DATA_INVALID",
65 => "CKR_ENCRYPTED_DATA_LEN_RANGE",
66 => "CKR_AEAD_DECRYPT_FAILED",
80 => "CKR_FUNCTION_CANCELED",
81 => "CKR_FUNCTION_NOT_PARALLEL",
84 => "CKR_FUNCTION_NOT_SUPPORTED",
96 => "CKR_KEY_HANDLE_INVALID",
98 => "CKR_KEY_SIZE_RANGE",
99 => "CKR_KEY_TYPE_INCONSISTENT",
100 => "CKR_KEY_NOT_NEEDED",
101 => "CKR_KEY_CHANGED",
102 => "CKR_KEY_NEEDED",
103 => "CKR_KEY_INDIGESTIBLE",
104 => "CKR_KEY_FUNCTION_NOT_PERMITTED",
105 => "CKR_KEY_NOT_WRAPPABLE",
106 => "CKR_KEY_UNEXTRACTABLE",
112 => "CKR_MECHANISM_INVALID",
113 => "CKR_MECHANISM_PARAM_INVALID",
130 => "CKR_OBJECT_HANDLE_INVALID",
144 => "CKR_OPERATION_ACTIVE",
145 => "CKR_OPERATION_NOT_INITIALIZED",
160 => "CKR_PIN_INCORRECT",
161 => "CKR_PIN_INVALID",
162 => "CKR_PIN_LEN_RANGE",
163 => "CKR_PIN_EXPIRED",
164 => "CKR_PIN_LOCKED",
176 => "CKR_SESSION_CLOSED",
177 => "CKR_SESSION_COUNT",
179 => "CKR_SESSION_HANDLE_INVALID",
180 => "CKR_SESSION_PARALLEL_NOT_SUPPORTED",
181 => "CKR_SESSION_READ_ONLY",
182 => "CKR_SESSION_EXISTS",
183 => "CKR_SESSION_READ_ONLY_EXISTS",
184 => "CKR_SESSION_READ_WRITE_SO_EXISTS",
192 => "CKR_SIGNATURE_INVALID",
193 => "CKR_SIGNATURE_LEN_RANGE",
208 => "CKR_TEMPLATE_INCOMPLETE",
209 => "CKR_TEMPLATE_INCONSISTENT",
224 => "CKR_TOKEN_NOT_PRESENT",
225 => "CKR_TOKEN_NOT_RECOGNIZED",
226 => "CKR_TOKEN_WRITE_PROTECTED",
240 => "CKR_UNWRAPPING_KEY_HANDLE_INVALID",
241 => "CKR_UNWRAPPING_KEY_SIZE_RANGE",
242 => "CKR_UNWRAPPING_KEY_TYPE_INCONSISTENT",
256 => "CKR_USER_ALREADY_LOGGED_IN",
257 => "CKR_USER_NOT_LOGGED_IN",
258 => "CKR_USER_PIN_NOT_INITIALIZED",
259 => "CKR_USER_TYPE_INVALID",
260 => "CKR_USER_ANOTHER_ALREADY_LOGGED_IN",
261 => "CKR_USER_TOO_MANY_TYPES",
272 => "CKR_WRAPPED_KEY_INVALID",
274 => "CKR_WRAPPED_KEY_LEN_RANGE",
275 => "CKR_WRAPPING_KEY_HANDLE_INVALID",
276 => "CKR_WRAPPING_KEY_SIZE_RANGE",
277 => "CKR_WRAPPING_KEY_TYPE_INCONSISTENT",
288 => "CKR_RANDOM_SEED_NOT_SUPPORTED",
289 => "CKR_RANDOM_NO_RNG",
304 => "CKR_DOMAIN_PARAMS_INVALID",
320 => "CKR_CURVE_NOT_SUPPORTED",
336 => "CKR_BUFFER_TOO_SMALL",
352 => "CKR_SAVED_STATE_INVALID",
368 => "CKR_INFORMATION_SENSITIVE",
384 => "CKR_STATE_UNSAVEABLE",
400 => "CKR_CRYPTOKI_NOT_INITIALIZED",
401 => "CKR_CRYPTOKI_ALREADY_INITIALIZED",
416 => "CKR_MUTEX_BAD",
417 => "CKR_MUTEX_NOT_LOCKED",
432 => "CKR_NEW_PIN_MODE",
433 => "CKR_NEXT_OTP",
437 => "CKR_EXCEEDED_MAX_ITERATIONS",
438 => "CKR_FIPS_SELF_TEST_FAILED",
439 => "CKR_LIBRARY_LOAD_FAILED",
440 => "CKR_PIN_TOO_WEAK",
441 => "CKR_PUBLIC_KEY_INVALID",
512 => "CKR_FUNCTION_REJECTED",
513 => "CKR_TOKEN_RESOURCE_EXCEEDED",
514 => "CKR_OPERATION_CANCEL_FAILED",
515 => "CKR_KEY_EXHAUSTED",
2147483648 => "CKR_VENDOR_DEFINED",
_ => "Unknown CK_RV value",
}
}
pub fn get_ck_attribute_name(value: CK_ATTRIBUTE_TYPE) -> &'static str {
match value {
0 => "CKA_CLASS",
1 => "CKA_TOKEN",
2 => "CKA_PRIVATE",
3 => "CKA_LABEL",
4 => "CKA_UNIQUE_ID",
16 => "CKA_APPLICATION",
17 => "CKA_VALUE",
18 => "CKA_OBJECT_ID",
128 => "CKA_CERTIFICATE_TYPE",
129 => "CKA_ISSUER",
130 => "CKA_SERIAL_NUMBER",
131 => "CKA_AC_ISSUER",
132 => "CKA_OWNER",
133 => "CKA_ATTR_TYPES",
134 => "CKA_TRUSTED",
135 => "CKA_CERTIFICATE_CATEGORY",
136 => "CKA_JAVA_MIDP_SECURITY_DOMAIN",
137 => "CKA_URL",
138 => "CKA_HASH_OF_SUBJECT_PUBLIC_KEY",
139 => "CKA_HASH_OF_ISSUER_PUBLIC_KEY",
140 => "CKA_NAME_HASH_ALGORITHM",
144 => "CKA_CHECK_VALUE",
256 => "CKA_KEY_TYPE",
257 => "CKA_SUBJECT",
258 => "CKA_ID",
259 => "CKA_SENSITIVE",
260 => "CKA_ENCRYPT",
261 => "CKA_DECRYPT",
262 => "CKA_WRAP",
263 => "CKA_UNWRAP",
264 => "CKA_SIGN",
265 => "CKA_SIGN_RECOVER",
266 => "CKA_VERIFY",
267 => "CKA_VERIFY_RECOVER",
268 => "CKA_DERIVE",
272 => "CKA_START_DATE",
273 => "CKA_END_DATE",
288 => "CKA_MODULUS",
289 => "CKA_MODULUS_BITS",
290 => "CKA_PUBLIC_EXPONENT",
291 => "CKA_PRIVATE_EXPONENT",
292 => "CKA_PRIME_1",
293 => "CKA_PRIME_2",
294 => "CKA_EXPONENT_1",
295 => "CKA_EXPONENT_2",
296 => "CKA_COEFFICIENT",
297 => "CKA_PUBLIC_KEY_INFO",
304 => "CKA_PRIME",
305 => "CKA_SUBPRIME",
306 => "CKA_BASE",
307 => "CKA_PRIME_BITS",
308 => "CKA_SUBPRIME_BITS",
352 => "CKA_VALUE_BITS",
353 => "CKA_VALUE_LEN",
354 => "CKA_EXTRACTABLE",
355 => "CKA_LOCAL",
356 => "CKA_NEVER_EXTRACTABLE",
357 => "CKA_ALWAYS_SENSITIVE",
358 => "CKA_KEY_GEN_MECHANISM",
368 => "CKA_MODIFIABLE",
369 => "CKA_COPYABLE",
370 => "CKA_DESTROYABLE",
384 => "CKA_ECDSA_PARAMS",
385 => "CKA_EC_POINT",
512 => "CKA_SECONDARY_AUTH",
513 => "CKA_AUTH_PIN_FLAGS",
514 => "CKA_ALWAYS_AUTHENTICATE",
528 => "CKA_WRAP_WITH_TRUSTED",
1073742353 => "CKA_WRAP_TEMPLATE",
1073742354 => "CKA_UNWRAP_TEMPLATE",
1073742355 => "CKA_DERIVE_TEMPLATE",
544 => "CKA_OTP_FORMAT",
545 => "CKA_OTP_LENGTH",
546 => "CKA_OTP_TIME_INTERVAL",
547 => "CKA_OTP_USER_FRIENDLY_MODE",
548 => "CKA_OTP_CHALLENGE_REQUIREMENT",
549 => "CKA_OTP_TIME_REQUIREMENT",
550 => "CKA_OTP_COUNTER_REQUIREMENT",
551 => "CKA_OTP_PIN_REQUIREMENT",
558 => "CKA_OTP_COUNTER",
559 => "CKA_OTP_TIME",
554 => "CKA_OTP_USER_IDENTIFIER",
555 => "CKA_OTP_SERVICE_IDENTIFIER",
556 => "CKA_OTP_SERVICE_LOGO",
557 => "CKA_OTP_SERVICE_LOGO_TYPE",
592 => "CKA_GOSTR3410_PARAMS",
593 => "CKA_GOSTR3411_PARAMS",
594 => "CKA_GOST28147_PARAMS",
768 => "CKA_HW_FEATURE_TYPE",
769 => "CKA_RESET_ON_INIT",
770 => "CKA_HAS_RESET",
1024 => "CKA_PIXEL_X",
1025 => "CKA_PIXEL_Y",
1026 => "CKA_RESOLUTION",
1027 => "CKA_CHAR_ROWS",
1028 => "CKA_CHAR_COLUMNS",
1029 => "CKA_COLOR",
1030 => "CKA_BITS_PER_PIXEL",
1152 => "CKA_CHAR_SETS",
1153 => "CKA_ENCODING_METHODS",
1154 => "CKA_MIME_TYPES",
1280 => "CKA_MECHANISM_TYPE",
1281 => "CKA_REQUIRED_CMS_ATTRIBUTES",
1282 => "CKA_DEFAULT_CMS_ATTRIBUTES",
1283 => "CKA_SUPPORTED_CMS_ATTRIBUTES",
1073743360 => "CKA_ALLOWED_MECHANISMS",
1537 => "CKA_PROFILE_ID",
1538 => "CKA_X2RATCHET_BAG",
1539 => "CKA_X2RATCHET_BAGSIZE",
1540 => "CKA_X2RATCHET_BOBS1STMSG",
1541 => "CKA_X2RATCHET_CKR",
1542 => "CKA_X2RATCHET_CKS",
1543 => "CKA_X2RATCHET_DHP",
1544 => "CKA_X2RATCHET_DHR",
1545 => "CKA_X2RATCHET_DHS",
1546 => "CKA_X2RATCHET_HKR",
1547 => "CKA_X2RATCHET_HKS",
1548 => "CKA_X2RATCHET_ISALICE",
1549 => "CKA_X2RATCHET_NHKR",
1550 => "CKA_X2RATCHET_NHKS",
1551 => "CKA_X2RATCHET_NR",
1552 => "CKA_X2RATCHET_NS",
1553 => "CKA_X2RATCHET_PNS",
1554 => "CKA_X2RATCHET_RK",
1559 => "CKA_HSS_LEVELS",
1560 => "CKA_HSS_LMS_TYPE",
1561 => "CKA_HSS_LMOTS_TYPE",
1562 => "CKA_HSS_LMS_TYPES",
1563 => "CKA_HSS_LMOTS_TYPES",
1564 => "CKA_HSS_KEYS_REMAINING",
2147483648 => "CKA_VENDOR_DEFINED",
_ => "Unknown CK_ATTRIBUTE_TYPE value",
}
}

3
crate/server/Cargo.toml
View file

@ -69,7 +69,7 @@ opentelemetry = "0.23.0"
opentelemetry-otlp = { version = "0.16.0", features = ["tonic"] }
opentelemetry-semantic-conventions = { version = "0.15.0" }
opentelemetry_sdk = { version = "0.23.0", features = ["rt-tokio"] }
proteccio_pkcs11_loader = { path = "../pkcs11/proteccio" }
proteccio_pkcs11_loader = { path = "../hsm/proteccio" }
# Important: align the rustls version with reqwest rustls dependency
# When using client certificate authentication, reqwest will use the
# native-tls crate to create an Identity; this will be different backend
@ -88,6 +88,7 @@ tracing = { workspace = true }
tracing-opentelemetry = "0.24.0"
tracing-subscriber = { workspace = true, features = ["env-filter"] }
url = { workspace = true }
utimaco_pkcs11_loader = { path = "../hsm/utimaco" }
uuid = { workspace = true, features = ["v4"] }
x509-parser = { workspace = true }
zeroize = { workspace = true }

4
crate/server/src/config/command_line/clap_config.rs
View file

@ -95,8 +95,8 @@ pub struct ClapConfig {
pub info: bool,
/// The HSM model.
/// Only `proteccio` is supported for now.
#[clap(verbatim_doc_comment, long,value_parser(["proteccio"]), default_value = "proteccio")]
/// Trustway Proteccio and Utimaco General purpose HSMs are supported.
#[clap(verbatim_doc_comment, long,value_parser(["proteccio", "utimaco"]), default_value = "proteccio")]
pub hsm_model: String,
/// The username of the HSM admin.

84
crate/server/src/core/kms/mod.rs
View file

@ -9,6 +9,8 @@ use cosmian_kms_server_database::Database;
#[cfg(all(target_os = "linux", target_arch = "x86_64"))]
use proteccio_pkcs11_loader::Proteccio;
use tokio::sync::RwLock;
#[cfg(all(target_os = "linux", target_arch = "x86_64"))]
use utimaco_pkcs11_loader::Utimaco;
use crate::{config::ServerParams, error::KmsError, kms_bail, result::KResult};
@ -41,37 +43,7 @@ impl KMS {
/// A new KMS instance.
#[allow(clippy::as_conversions)]
pub(crate) async fn instantiate(server_params: ServerParams) -> KResult<Self> {
// Instantiate the HSM if any; the code has support for multiple concurrent HSMs
let hsm: Option<Arc<dyn HSM + Send + Sync>> = if server_params.slot_passwords.is_empty() {
None
} else {
if server_params
.hsm_model
.as_ref()
.map(String::from)
.unwrap_or_default()
!= "proteccio"
{
kms_bail!("The only supported HSM model is Proteccio for now")
}
#[cfg(not(all(target_os = "linux", target_arch = "x86_64")))]
kms_bail!("Fatal: Proteccio HSM is only supported on Linux x86_64");
#[cfg(all(target_os = "linux", target_arch = "x86_64"))]
{
let proteccio: Arc<dyn HSM + Send + Sync> = Arc::new(
Proteccio::instantiate(
"/lib/libnethsm.so",
server_params.slot_passwords.clone(),
)
.map_err(|e| {
KmsError::InvalidRequest(format!(
"Failed to instantiate the Proteccio HSM: {e}"
))
})?,
);
Some(proteccio)
}
};
let hsm = Self::instantiate_hsm(&server_params)?;
// Instantiate the main database
let main_db_params = server_params.main_db_params.as_ref().ok_or_else(|| {
@ -108,4 +80,54 @@ impl KMS {
encryption_oracles: RwLock::new(encryption_oracles),
})
}
fn instantiate_hsm(
server_params: &ServerParams,
) -> Result<Option<Arc<dyn HSM + Send + Sync>>, KmsError> {
// Instantiate the HSM if any; the code has support for multiple concurrent HSMs
let hsm: Option<Arc<dyn HSM + Send + Sync>> = if server_params.slot_passwords.is_empty() {
None
} else {
#[cfg(not(all(target_os = "linux", target_arch = "x86_64")))]
kms_bail!("Fatal: HSMs are only supported on Linux x86_64");
#[cfg(all(target_os = "linux", target_arch = "x86_64"))]
{
let hsm_model = server_params.hsm_model.as_ref().ok_or_else(|| {
KmsError::InvalidRequest("The HSM model is not specified".to_owned())
})?;
match hsm_model.as_str() {
"proteccio" => {
let proteccio: Arc<dyn HSM + Send + Sync> = Arc::new(
Proteccio::instantiate(
"/lib/libnethsm.so",
server_params.slot_passwords.clone(),
)
.map_err(|e| {
KmsError::InvalidRequest(format!(
"Failed to instantiate the Proteccio HSM: {e}"
))
})?,
);
Some(proteccio)
}
"utimaco" => {
let utimaco: Arc<dyn HSM + Send + Sync> = Arc::new(
Utimaco::instantiate(
"/lib/libcs_pkcs11_R3.so",
server_params.slot_passwords.clone(),
)
.map_err(|e| {
KmsError::InvalidRequest(format!(
"Failed to instantiate the Utimaco HSM: {e}"
))
})?,
);
Some(utimaco)
}
_ => kms_bail!("The only supported HSM models are proteccio and utimaco"),
}
}
};
Ok(hsm)
}
}

11
crate/server/src/core/operations/encrypt.rs
View file

@ -209,13 +209,10 @@ async fn encrypt_using_encryption_oracle(
request.authenticated_encryption_additional_data.as_deref(),
)
.await?;
let ciphertext_len = encrypted_content.ciphertext.len();
debug!("Encrypted using oracle: algorithm: {ca:?}, ciphertext length: {ciphertext_len}");
if ciphertext_len < 28 {
return Err(KmsError::CryptographicError(
"Encrypt: encryption oracle returned invalid ciphertext".to_owned(),
))
}
debug!(
"Encrypted using oracle: algorithm: {ca:?}, ciphertext length: {}",
encrypted_content.ciphertext.len()
);
Ok(EncryptResponse {
unique_identifier: UniqueIdentifier::TextString(uid.to_owned()),

296
documentation/docs/database.md
View file

@ -1,68 +1,94 @@
# Database
By default the server runs using a [SQLite](https://www.sqlite.org/) database, but it can be configured to use a choice
of databases: SQLite encrypted, [PostgreSQL](https://www.postgresql.org/), [Maria DB](https://mariadb.org/),
and [MySQL](https://www.mysql.com/), as well as [Redis](https://redis.io/), using
the [Redis-with-Findex](#redis-with-findex)
configuration.
<!-- TOC -->
- [Selecting the database](#selecting-the-database)
- [Redis with Findex](#redis-with-findex)
- [Configuring the database](#configuring-the-database)
- [SQLite](#sqlite)
- [PostgreSQL](#postgresql)
- [MySQL or MariaDB](#mysql-or-mariadb)
- [Redis with Findex](#redis-with-findex-1)
- [SQLite encrypted](#sqlite-encrypted)
- [Clearing the database](#clearing-the-database)
- [Database migration](#database-migration)
<!-- TOC -->
## Selecting the database
The KMS server has support for PostgreSQL, Maria DB, and MySQL, as well as Redis, using the Redis-with-Findex configuration.
All databases but SQLite and SQLite encrypted can be used in a high-availability setup.
Redis with Findex offers the ability to use Redis as a database with application-level encryption: all data is encrypted (using AES 256 GCM) by the KMS servers before being sent to Redis. [Findex](https://github.com/Cosmian/findex/) is a Cosmian cryptographic algorithm used to build encrypted indexes on encrypted data, also stored in Redis. This allows the KMS to perform fast encrypted queries on encrypted data. Redis with Findex offers post-quantum resistance on encrypted data and encrypted indexes.
The **SQLite** database can serve high loads and millions of objects, and is very suitable
for scenarios that do not demand high availability. To use SQLIte encrypted, see
the [SQLite encrypted](#sqlite-encrypted) section.
Redis-with-Findex is most useful when:
### Redis with Findex
- KMS servers are run inside a confidential VM or an enclave. In this case, the secret used to encrypt the Redis data and indexes is protected by the VM or enclave and cannot be recovered at runtime by inspecting the KMS servers' memory.
**Redis with Findex** offers the ability to use Redis as a database with application-level encryption: all data is
encrypted (using AES 256 GCM) by the KMS servers before being sent to
Redis. [Findex](https://github.com/Cosmian/findex/) is a Cosmian cryptographic algorithm used to build encrypted indexes
on encrypted data, also stored in Redis. This allows the KMS to perform fast encrypted queries on encrypted data. Redis
with Findex offers post-quantum resistance on encrypted data and encrypted indexes.
**Redis-with-Findex** is most useful when:
- KMS servers are run inside a confidential VM or an enclave. In this case, the secret used to encrypt the Redis data
and indexes, is protected by the VM or enclave and cannot be recovered at runtime by inspecting the KMS servers'
memory.
- KMS servers are run by a trusted party but the Redis backend is managed by an untrusted third party.
Redis-with-Findex should be selected to [run the Cosmian KMS in the cloud or any other zero-trust environment](./marketplace_guide.md).
Redis-with-Findex is the database selected
to [run the Cosmian KMS in the cloud or any other zero-trust environment](installation/marketplace_guide.md).
## Configuring the database
The database parameters may be configured either:
- using options on the command line that is used to start the KMS server
- the [TOML configuration file](./server_configuration_file.md)
- or the [arguments passed to the server](./server_cli.md) on the command line.
### Configuring the database via the command line
### SQLite
For
This is the default configuration. To use SQLite, no additional configuration is needed.
- PostgreSQL, use:
=== "kms.toml"
```sh
docker run --rm -p 9998:9998 \
--name kms ghcr.io/cosmian/kms:latest \
--database-type=postgresql \
--database-url=postgres://kms_user:kms_password@pgsql-server:5432/kms
```
```toml
[db]
database_type = "sqlite"
sqlite_path = "./sqlite-data"
```
- MySQL or MariaDB, use:
=== "Command line arguments"
```sh
docker run --rm -p 9998:9998 \
--name kms ghcr.io/cosmian/kms:latest \
--database-type=mysql \
--database-url=mysql://kms_user:kms_password@mariadb:3306/kms
```
```sh
--database-type=sqlite \
--sqlite-path="./sqlite-data"
```
- Redis (with-Findex), use:
#### PostgreSQL
For Redis with Findex, the `--redis-master-password` and `--redis-findex-label` options must also be specified:
=== "kms.toml"
- the `redis-master-password` is the password from which keys will be derived (using Argon 2) to encrypt the Redis data and indexes.
- the `redis-findex-label` is a public arbitrary label that can be changed to rotate the Findex ciphertexts without changing the password/key.
```toml
[db]
database-type="postgresql"
database-url="postgres://kms_user:kms_password@pgsql-server:5432/kms"
```
```sh
docker run --rm -p 9998:9998 \
--name kms ghcr.io/cosmian/kms:latest \
--database-type=redis-findex \
--database-url=redis://localhost:6379 \
--redis-master-password password \
--redis-findex-label label
```
=== "Command line arguments"
The `redis-master-password` is the password from which a key will be derived (using Argon 2) to encrypt the Redis data and indexes.
The `redis-findex-label` is a public arbitrary label that can be changed to rotate the Findex ciphertexts without changing the password/key.
```sh
--database-type=postgresql \
--database-url=postgres://kms_user:kms_password@pgsql-server:5432/kms
```
!!!info "Setting up a PostgreSQL database"
Before running the server, a dedicated database with a dedicated user should be created on the PostgreSQL instance. These sample instructions create a database called `kms` owned by a user `kms_user` with password `kms_password`:
Before running the server, a dedicated database with a dedicated user should be created on the PostgreSQL instance.
These sample instructions create a database called `kms` owned by a user `kms_user` with password `kms_password`:
1. Connect to psql under user `postgres`
```sh
@ -79,42 +105,188 @@ The `redis-findex-label` is a public arbitrary label that can be changed to rota
create database kms owner=kms_user;
```
## Using a certificate to authenticate to MySQL or Maria DB
#### MySQL or MariaDB
Use a certificate to authenticate to MySQL or Maria DB with the `--mysql-user-cert-file` option on the command line. Specify the certificate file name and mount the file to docker.
=== "kms.toml"
Say the certificate is called `cert.p12` and is in a directory called `/certificate` on the host disk.
```toml
[db]
database-type="mysql"
database-url="mysql://kms_user:kms_password@mysql-server:3306/kms"
```
```sh
docker run --rm -p 9998:9998 \
--name kms ghcr.io/cosmian/kms:latest \
-v /certificate/cert.p12:/root/cosmian-kms/cert.p12 \
--database-type=mysql \
--database-url=mysql://mysql_server:3306/kms \
--mysql-user-cert-file=cert.p12
```
=== "Command line arguments"
```sh
--database-type=mysql \
--database-url=mysql://kms_user:kms_password@mariadb:3306/kms
```
!!!info "Using a certificate to authenticate to MySQL or Maria DB"
Use a certificate to authenticate to MySQL or Maria DB with the `mysql-user-cert-file` option to
specify the certificate file name.
**Docker Example**: say the certificate is called `cert.p12`
and is in a directory called `/certificate` on the host disk.
```sh
docker run --rm -p 9998:9998 \
--name kms ghcr.io/cosmian/kms:latest \
-v /certificate/cert.p12:/root/cosmian-kms/cert.p12 \
--database-type=mysql \
--database-url=mysql://mysql_server:3306/kms \
--mysql-user-cert-file=cert.p12
```
#### Redis with Findex
For Redis with Findex, the `--redis-master-password` and `--redis-findex-label` options must also be specified:
- the `redis-master-password` is the password from which keys will be derived (using Argon 2) to encrypt the Redis data
and indexes.
- the `redis-findex-label` is a public arbitrary label that can be changed to rotate the Findex ciphertexts without
changing the password/key.
=== "kms.toml"
```toml
[db]
database-type="redis-findex"
database-url="redis://localhost:6379"
redis-master-password="password"
redis-findex-label="label"
```
=== "Command line arguments"
```sh
--database-type=redis-findex \
--database-url=redis://localhost:6379 \
--redis-master-password=password \
--redis-findex-label=label
```
- Redis (with-Findex), use:
#### SQLite encrypted
=== "kms.toml"
```toml
[db]
database_type = "sqlite-enc"
sqlite_path = "./sqlite-data"
```
=== "Command line arguments"
```sh
--database-type=sqlite-enc \
--sqlite-path="./sqlite-data"
```
It requires now to install the [Cosmian CLI](../cosmian_cli/index.md) and create a new encrypted database.
!!! important "Important: encrypted databases must be created first"
Before using an encrypted database, you must create it by either using the [Cosmian CLI](../cosmian_cli/index.md)
or calling the `POST /new_database` endpoint.
The call will return a secret
=== "cosmian CLI"
```sh
➜ cosmian kms new-database
eyJncm91cF9pZCI6MzE0ODQ3NTQzOTU4OTM2Mjk5OTY2ODU4MTY1NzE0MTk0MjU5NjUyLCJrZXkiOiIzZDAyNzg3YjUyZGY5OTYzNGNkOTVmM2QxODEyNDk4YTRiZWU1Nzc1NmM5NDI0NjdhZDI5ZTYxZjFmMmM0OWViIn0=
```
=== "curl"
```sh
➜ curl -X POST https://my-server:9998/new_database
"eyJncm91cF9pZCI6MzE0ODQ3NTQzOTU4OTM2Mjk5OTY2ODU4MTY1NzE0MTk0MjU5NjUyLCJrZXkiOiIzZDAyNzg3YjUyZGY5OTYzNGNkOTVmM2QxODEyNDk4YTRiZWU1Nzc1NmM5NDI0NjdhZDI5ZTYxZjFmMmM0OWViIn0="%
```
The secret is the value between the quotes `""`.
Warning:
- This secret is only displayed **once** and is **not stored**
anywhere on the server.
- Each call to `new_database` will create a **new additional** database.
It will not return the secret of the last created database,
and it will not overwrite the last created database.
Once an encrypted database is created, the secret must be passed in every subsequent query to the
KMS server.
Passing the correct secret "auto-selects" the correct encrypted database: multiple encrypted
databases can be used concurrently on the same KMS server.
=== "cosmian CLI"
The secret must be set in `database_secret` property of the CLI `cosmian.json` configuration file,
and it will be used for all subsequent calls to the KMS server.
```toml
[kms_config.http_config]
server_url = "http://127.0.0.1:9990"
access_token = "eyJhbGciOiJSUzI1NiIsInR5cCI6Ik...yaJbDDql3A"
database_secret = "eyJncm91cF9pZCI6MTI5N...MWIwYjE5ZmNlN2U3In0="
```
=== "curl"
The secret must be passed using a `DatabaseSecret` HTTP header, e.g.
```sh
curl \
-H "DatabaseSecret: eyJncm91cF9pZCI6MzE0ODQ3NTQzOTU4OTM2Mjk5OTY2ODU4MTY1NzE0MTk0MjU5NjUyLCJrZXkiOiIzZDAyNzg3YjUyZGY5OTYzNGNkOTVmM2QxODEyNDk4YTRiZWU1Nzc1NmM5NDI0NjdhZDI5ZTYxZjFmMmM0OWViIn0=" \
http://localhost:9998/objects/owned
```
## Clearing the database
The KMS server can be configured to automatically clear the database on restart.
!!! warning "Warning: this operation is irreversible"
The cleanup operation will delete all objects and keys stored in the database.
=== "kms.toml"
```toml
[db]
cleanup_on_startup = true
```
=== "Command line arguments"
```sh
--cleanup-on-startup
```
## Database migration
Depending on the KMS database evolution, a migration can happen between 2 versions of the KMS server. It will be clearly written in the CHANGELOG.md. In that case, a generic database upgrade mechanism is run on startup.
Depending on the KMS database evolution, a migration can happen between 2 versions of the KMS server. It will be clearly
written in the CHANGELOG.md. In that case, a generic database upgrade mechanism is run on startup.
At first, the table `context` is responsible for storing the version of the software run and the state of the database. The state can be one of the following:
At first, the table `context` is responsible for storing the version of the software run and the state of the database.
The state can be one of the following:
- `ready`: the database is ready to be used
- `upgrading`: the database is being upgraded
On server startup:
On startup, the server checks if the software version is greater than the last version run:
- the server checks if the software version is greater than the last version run:
- if no, it simply starts;
- if yes:
- if no, it simply starts;
- if yes:
- it looks for all upgrades to apply in order from the last version run to this version;
- if there is any to run, it sets an upgrading flag on the db state field in the context table;
- it runs all the upgrades in order;
- it sets the flag from upgrading to ready;
- it looks for all upgrades to apply in order from the last version run to this version;
- if there is any to run, it sets an upgrading flag on the db state field in the context table;
- it runs all the upgrades in order;
- it sets the flag from upgrading to ready;
On every call to the database, a check is performed on the db state field to check if the database is upgrading. If yes, calls fail.
On every call to the database, a check is performed on the db state field to check if the database is upgrading. If yes,
calls fail.
Upgrades resist to being interrupted in the middle and resumed from start if that happens.

322
documentation/docs/drawings/RAG.drawio.svg Normal file
View file

@ -0,0 +1,322 @@
<svg host="65bd71144e" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" width="780px" height="679px" viewBox="-0.5 -0.5 780 679" content="&lt;mxfile&gt;&lt;diagram id=&quot;C0gcMnsRfcVC1bxaHtLa&quot; name=&quot;Page-1&quot;&gt;5VpZb+M2EP41BrYPCXTZkR8d5+hiE+xivU2faYm22VCiQVFJ3F/f4SHrIJ3YieLWKBbrSEOKHH5zjzQIp9nLLUfr1T1LMR0EXvoyCK8GQeD7oxD+SMpGU4JgFGjKkpPUzKoJM/I3NkTPUEuS4qI1UTBGBVm3iQnLc5yIFg1xzp7b0xaMtnddoyW2CLMEUZv6J0nFSlPjoVfTf8dkuap29j0zkqFqsiEUK5Sy5wYpvB6EU86Y0FfZyxRTiV6Fi37uZsfoljGOc7HPAwb3J0RLc7af17NfQJn8+Go4FJvq2MDsWl7iF1j7MmXlXI34cMNZmac4NXfPKyLwbI0SOfwMSgC0lcioGbaZNHw/YS7wS4NkmL7FLMOCb2BKNXphADQqFMaRvn+u5RGMzZxVQxbhyKiBUYHldukaJbgwQLlBCy3QJqVYwWFIggRhOYzdoxyUIJMHPCEUo9A7HoqRheJNSank7jpP+GYtAIhgRCVIcw5XS3n1AObM4M67QgLNC3xS6MZ7ojvyPo7u0EJ3yrIM5YCpd0dyAM77o8ASya+5wHwhcTghLId7QjnsAcqRBeUpIbWNPW9B5fehdr6/P1gapomMx0dBJorayAR+fD60sPGHDmyCuAdo7LBxc/1wdWk7uSkl//XAMe5CGTigDFxQRodrmXRU3+d/yUwu8CiaY2pSlcktECq0wK3NOTILeWsKx19BYod5O9nLUWbwfECcIImhom/FFDVObkTYEILQ6DfgLARnj3jKKMSl8Cpn4FvDywWhtENClCxzuE2wdLhAkHBDxkAnZiAjaSq3ccqPwewFVaniCubhvB9BBh2bGLvk6DSJd3gLuG2IcoeV2IHrOpvjNCX5UuZVupg4HcvwIweeQT92YYN3QKhqe9+UcJCJzF3Dq4KVktMjgNVQt8qRjOxoVSHYClb+RfVkD46kkXB6OsMsbK+scyeb3nz4HguUwj+BPuiF4v+VExpaahDH5xcuw3Gowmc5otiypW940ynturow2xQCZ5bNHeSEUlSstnPlzQ8kQFZSOL5cF356QX3YjeGORNHpqXrJE8c70H1QBBtYsMndBUqyoQQA5uHb6M61KO7mWwJKHpdKQN9LQWVlZCxKd50gHvWj490CxlEMjh3aHfcAdrBHUn6QijocS9sBDYLQ80aj6bSnoDpsu4fQESVCl6pudbqHIDHD/EkVzDOSOjR0sl5T2f/hqgH0IeevxXMS3n/BcvFJQh/5LZlHw5El89gh8ne0pvYJB46m34nZUNS2ochhQpHLhHos2Lal2ueb0OlUcZ9oQsNuLRL8qyZkl3aVDpS00oGKQklF+bWSzUpd6xPIy70MSeXx2AJ+5HoJy9YgnFwAuxM5oLqbSaVrhdK16XagqPy4HDiv9oMD1FvuYqKzZr1zxZpo8LqdTKu+BDCg+tuqga2nWk3ZRjfW+yK1Zq0L39dW/u2wU2gAznadoRBIYIoLea1ZS5B8pTFXz8qXYJI23ygYsrW0PI9jbblguO/hpUYUwCAlWL3AhQNv3Ci1dJFlOMyxXiXFaoIRRXYYL9/uZ3LBRD2O1A/8f1RpadZI+r3CZPlmc8OVfvHDOEoUJF/gwaLBGsXoCW8FCXsdKLUuFoKjfMF4VqudcmogxFww5SVMHeuVRVOFcKObkhHdTpnWp6iXsxe4kUl2rRatXRVe4KxEZZl6NFMFsZIUrLLMlcoTQERtticAE+VGzDJdjSG1opw51MSpHzh9HfuBfBVVEzth3gSPfSNGI8i4QsLO+HHkWBHF8R69GGeKsE29P1Sk2L2rWQnOUfF4jLafBZUD0J3ohUEbOz+KLex8/5PafsHF27lphZU05R7afh8CKwg7YDle3oeuVLSHt86B3dZ5B1Y540fCKhx1sbIVy4lV8HGsqnUbWN0x6eD980CnWj9hSYJhHK4n5VLGR+V1b3GOufkWojfvKZ2f+RYI0ntzbxb2X/OkVc5O8UL0I5XhuCWUM99h7V4/1g639edAaqzxVVV4/Q8=&lt;/diagram&gt;&lt;/mxfile&gt;">
<defs/>
<g>
<rect x="120" y="402" width="290" height="36" rx="5.4" ry="5.4" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<rect x="122" y="404" width="286" height="32" rx="4.8" ry="4.8" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 284px; height: 1px; padding-top: 420px; margin-left: 123px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
REST API
</div>
</div>
</div>
</foreignObject>
<text x="265" y="424" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
REST API
</text>
</switch>
</g>
<rect x="120" y="448" width="290" height="36" rx="5.4" ry="5.4" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<rect x="122" y="450" width="286" height="32" rx="4.8" ry="4.8" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 284px; height: 1px; padding-top: 466px; margin-left: 123px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
Authentication Management
</div>
</div>
</div>
</foreignObject>
<text x="265" y="470" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
Authentication Management
</text>
</switch>
</g>
<rect x="120" y="498" width="290" height="60" rx="9" ry="9" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<rect x="122" y="500" width="286" height="56" rx="8.4" ry="8.4" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 284px; height: 1px; padding-top: 528px; margin-left: 123px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
Fully Encrypted
<br/>
Vector Databse
</div>
</div>
</div>
</foreignObject>
<text x="265" y="532" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
Fully Encrypted...
</text>
</switch>
</g>
<rect x="120" y="68" width="290" height="50" rx="7.5" ry="7.5" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<rect x="122" y="70" width="286" height="46" rx="6.9" ry="6.9" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 284px; height: 1px; padding-top: 93px; margin-left: 123px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
Command Line User Interface
</div>
</div>
</div>
</foreignObject>
<text x="265" y="97" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
Command Line User Interface
</text>
</switch>
</g>
<rect x="120" y="118" width="290" height="160" rx="24" ry="24" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<rect x="122" y="120" width="286" height="156" rx="23.4" ry="23.4" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<path d="M 390 250.5 L 420 236.5 L 420 246.3 L 510 246.3 L 510 236.5 L 540 250.5 L 510 264.5 L 510 254.7 L 420 254.7 L 420 264.5 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/>
<rect x="140" y="230.5" width="250" height="40" rx="6" ry="6" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<rect x="142" y="232.5" width="246" height="36" rx="5.4" ry="5.4" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 244px; height: 1px; padding-top: 251px; margin-left: 143px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
FEVDB
<br/>
Client
</div>
</div>
</div>
</foreignObject>
<text x="265" y="254" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
FEVDB...
</text>
</switch>
</g>
<rect x="190" y="110.5" width="150" height="60" fill="none" stroke="none" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 148px; height: 1px; padding-top: 141px; margin-left: 191px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center; max-height: 56px; overflow: hidden;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
RAG Client Library
</div>
</div>
</div>
</foreignObject>
<text x="265" y="144" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
RAG Client Library
</text>
</switch>
</g>
<rect x="140" y="163" width="120" height="40" rx="6" ry="6" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<rect x="142" y="165" width="116" height="36" rx="5.4" ry="5.4" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 114px; height: 1px; padding-top: 183px; margin-left: 143px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
Embedding Model
</div>
</div>
</div>
</foreignObject>
<text x="200" y="187" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
Embedding Model
</text>
</switch>
</g>
<path d="M 206.25 336.75 L 229.75 314.25 L 229.75 330 L 300.25 330 L 300.25 314.25 L 323.75 336.75 L 300.25 359.25 L 300.25 343.5 L 229.75 343.5 L 229.75 359.25 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" transform="rotate(90,265,336.75)" pointer-events="all"/>
<rect x="202.5" y="306.75" width="125" height="60" fill="none" stroke="none" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 123px; height: 1px; padding-top: 337px; margin-left: 204px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center; max-height: 56px; overflow: hidden;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
Encrypted Vectors
<br/>
and
<br/>
Encrypted Metadadata
</div>
</div>
</div>
</foreignObject>
<text x="265" y="340" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
Encrypted Vectors...
</text>
</switch>
</g>
<rect x="540" y="218" width="120" height="60" rx="9" ry="9" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-dasharray="12 12" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 248px; margin-left: 541px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
Key Management
<br/>
System
</div>
</div>
</div>
</foreignObject>
<text x="600" y="252" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
Key Management...
</text>
</switch>
</g>
<path d="M 220 613 C 220 604.72 241.27 598 267.5 598 C 280.1 598 292.18 599.58 301.09 602.39 C 310 605.21 315 609.02 315 613 L 315 663 C 315 671.28 293.73 678 267.5 678 C 241.27 678 220 671.28 220 663 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/>
<path d="M 315 613 C 315 621.28 293.73 628 267.5 628 C 241.27 628 220 621.28 220 613" fill="none" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 93px; height: 1px; padding-top: 651px; margin-left: 221px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
Key Value
<br/>
Store
</div>
</div>
</div>
</foreignObject>
<text x="268" y="654" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
Key Value...
</text>
</switch>
</g>
<rect x="105" y="378" width="320" height="200" rx="30" ry="30" fill="none" stroke="#0066cc" pointer-events="all"/>
<rect x="11" y="474" width="80" height="36" fill="none" stroke="none" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 78px; height: 1px; padding-top: 492px; margin-left: 12px;">
<div data-drawio-colors="color: #0066CC; " style="box-sizing: border-box; font-size: 0px; text-align: center; max-height: 32px; overflow: hidden;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 102, 204); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
Server Side
<br/>
Applicatrion
</div>
</div>
</div>
</foreignObject>
<text x="51" y="496" fill="#0066CC" font-family="Helvetica" font-size="12px" text-anchor="middle">
Server Side...
</text>
</switch>
</g>
<rect x="95" y="58" width="340" height="240" rx="36" ry="36" fill="none" stroke="#0066cc" pointer-events="all"/>
<rect x="0" y="160" width="80" height="36" fill="none" stroke="none" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 78px; height: 1px; padding-top: 178px; margin-left: 1px;">
<div data-drawio-colors="color: #0066CC; " style="box-sizing: border-box; font-size: 0px; text-align: center; max-height: 32px; overflow: hidden;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 102, 204); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
Client Side
<br/>
Applicatrion
</div>
</div>
</div>
</foreignObject>
<text x="40" y="182" fill="#0066CC" font-family="Helvetica" font-size="12px" text-anchor="middle">
Client Side...
</text>
</switch>
</g>
<rect x="438.5" y="278" width="340" height="320" fill="none" stroke="none" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe flex-start; width: 338px; height: 1px; padding-top: 438px; margin-left: 441px;">
<div data-drawio-colors="color: #0066CC; " style="box-sizing: border-box; font-size: 0px; text-align: left; max-height: 316px; overflow: hidden;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 102, 204); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
<ul>
<li>
The RAG is made of 2 components: one client side, one server side.
</li>
<li>
The client side component is the RAG client library only or the Command Line Interface (wrapping the RAG client library).
</li>
<li>
The server-side component is stateless and can be scaled by simple replication.
</li>
<li>
The server side maniuplates client side encrypted data and never decrypts them.
</li>
<li>
The KMS acts as a key management system and encryption oracle (keys never leave the KMS).
</li>
<li>
The client side tranforms the text into vectors using the embedding miodel, encrypts the vectors using Findes and the text as part of the metadata usign Coivercrypt
</li>
<li>
All data server side is client-side encrypted and never decrypted.
</li>
</ul>
</div>
</div>
</div>
</foreignObject>
<text x="441" y="442" fill="#0066CC" font-family="Helvetica" font-size="12px">
The RAG is made of 2 components: one client side, one se...
</text>
</switch>
</g>
<rect x="275" y="166" width="110" height="40" rx="6" ry="6" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<rect x="277" y="168" width="106" height="36" rx="5.4" ry="5.4" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 104px; height: 1px; padding-top: 186px; margin-left: 278px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
Summary Model
</div>
</div>
</div>
</foreignObject>
<text x="330" y="190" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
Summary Model
</text>
</switch>
</g>
<path d="M 187 210.5 L 207.8 210.5 L 207.8 200 L 213 215 L 207.8 230 L 207.8 219.5 L 187 219.5 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" transform="rotate(90,200,215)" pointer-events="all"/>
<path d="M 319 212.5 L 336.6 212.5 L 336.6 202 L 341 217 L 336.6 232 L 336.6 221.5 L 319 221.5 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" transform="rotate(270,330,217)" pointer-events="all"/>
<rect x="9" y="0" width="100" height="40" fill="none" stroke="none" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe flex-start; width: 1px; height: 1px; padding-top: 20px; margin-left: 11px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: left;">
<div style="display: inline-block; font-size: 24px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; font-weight: bold; white-space: nowrap;">
Lot 1.2: Retrieval Augmented Generation
</div>
</div>
</div>
</foreignObject>
<text x="11" y="27" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="24px" font-weight="bold">
Lot 1.2:...
</text>
</switch>
</g>
</g>
<switch>
<g requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"/>
<a transform="translate(0,-5)" xlink:href="https://www.diagrams.net/doc/faq/svg-export-text-problems" target="_blank">
<text text-anchor="middle" font-size="10px" x="50%" y="100%">
Text is not SVG - cannot display
</text>
</a>
</switch>
</svg>
Side by side

After

Width:  |  Height:  |  Size: 28 KiB

520
documentation/docs/drawings/ai_cse_1_of_2.drawio.svg
View file

File diff suppressed because one or more lines are too long
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 696 KiB

After

Width:  |  Height:  |  Size: 578 KiB

Before After
Before After

0
documentation/docs/drawings/ai_disk_encryption_aes_xts.svg → documentation/docs/drawings/ai_disk_encryption_aes_xts.drawio.svg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 694 KiB

After

Width:  |  Height:  |  Size: 694 KiB

Before After
Before After

0
documentation/docs/drawings/cse_schema.svg → documentation/docs/drawings/cse_schema.drawio.svg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 380 KiB

After

Width:  |  Height:  |  Size: 380 KiB

Before After
Before After

324
documentation/docs/drawings/fully_encrypted_vector_database.drawio.svg Normal file
View file

@ -0,0 +1,324 @@
<svg host="65bd71144e" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" width="769px" height="711px" viewBox="-0.5 -0.5 769 711" content="&lt;mxfile&gt;&lt;diagram id=&quot;C0gcMnsRfcVC1bxaHtLa&quot; name=&quot;Page-1&quot;&gt;5Vpbc+I2FP41zKQPyfgCDjyyhLSZJt2dsk37KmyB1ciWK8tJ6K/v0cXGRiKBBLJhOjtL7KPb0XfuB3rhJHv+maMivWMJpr3AS5574VUvCHw/CuGPpKw0JQiiQFOWnCRm1powI/9iQ/QMtSIJLjsTBWNUkKJLjFme41h0aIhz9tSdtmC0e2qBltgizGJEbeqfJBGppg4H3pr+CybLtD7Z98xIhurJhlCmKGFPLVI47YUTzpjQT9nzBFOJXo2LXne9ZbRhjONc7LLA4P6IaGXu9vt09h0o4283hkOxqq8NzBbyET/D3l8SVs3ViA8vnFV5ghPz9pQSgWcFiuXwEygB0FKRUTNszsRc4OetfPsNGqBHmGVY8BVMqRdcGgCNCoXDvn5/WssjGJk5aUsWtYyQUYFls/UaJXgwQLlBCy3QxpVIgXMSI0FY3gsiKhGac3hayqc7lINOZPJyJwRqv2+D6vcdoEYHALVvgXp989vV9C8bzBnmcE2YOwMvAH+m+ZLk+DMDGwY/ENiBBeyEZRnKEyDeKuC8P0qF500uMF9IHD4xlptKWhv0Rxh+ZEF5Skg1Ueg1qJqJ78HK93cHS8M0lpH5Q5Dp94cXgy42g8i2yIEDm2B4AGjsALLN100oUUHjdHydP9rQOoeB+nUIbSPb31/ppN/6Ov9bpniBR9EcU+PgatRuyZwjs4VXULh2Crke5t38L0eZwfEecYIkdoreyKvfurGRZQt8oVFvwVgKzh7whFHGgZIzkFj4ZUEo3SAhSpY5vMZYel4gSOQhiaBjM5CRJJHHOOXGYPaCquwxhXk4P4wAG0EYAY6C2lZes403RCt4bQlxi7nYEezXu5nDVnS2DwB/YvMIg4Hleo5nIDaWe4SwrldOCAdwZXYbXpWskpx+AFwt7avhGl069LE/cISx4eE8yjSP+aoQWCZN90rJSlsBdU5l09uL77BACfwT6J1Oafi/8kmDTTUIBi418AOHHhzLLQ1tt4Qlz+1iz6pdVqXAmWVye3mhBJVpM1e+fEMCRCVl48t94eMgoA9GVpYUukE/UuXij7YgfK8INrhglttzo3hFCYDMw9cRnmtx3M4bAooflkpIXytBZQZmjEq3piBCHUbNo8sNxCNHzj5y6PgbXJ3dC/Jehtsz+HpXnKgC/PNGWbtIHFkwHq1KDHaofPYCxOGlu968F4SeF0WTyYHQi7rohZe2EoYuow/eUDhui7gbXZ5NWx8XBZXtNq76be+KpFo8JxFKFywXRxJ65Hdk3ncUxEOHyMPoKMHV0WM9MRsadB25M3I6jajxXYcshI9vRKdTIx/TiMJuvuSFP9SI7MK51oGK1jpQUyipKd9TGeSvK0rlzezKR2YAUL/MUSnnESiEvAyp7hRbwIc8MWZZAeLLBVxoLAdUmzmutbFU2jhpBsra18uBi5ojuOKaqW1sbuy5PrlmTXSn0bojBEer+6n76ElWX7zVEPfOpEYVJF9u3fOn/TjXlz7fxncpkMAUl/JZMxUjMFJvrtbK7yIlbb5SV88KaY8ex9qewZzfwssaRYCBVOALBC4dGOOWSuhK1nCYY71LgtUEA3+2Hy+qt+OhWC1H6gP+P6hMNGuVVl5pailzuOFKuTKQKooVJGewsGyxRjF6xI0I4aydpTbt7E8WzSaPdT9AC64AlWE8U+ho5GoZV6VWn2tZhzy/7dz1sZnqJCj0Xz530jp6Ih2i2k/T5eqCleL8nwrlosqUEpUElE9BjOiScSLSTPkwcGEAczbHSaJOQXGs9bNgoHZE6cpZCnKQ7BWcPZJEn2rYlC6ck1jpW1zBvq+ZDBDbjmoj/puYsmsgacUeV6TYGlY+OIT0h5tVN+xvZw6ubwubdOJd1YvdIbxlUhf8C1878x0jw8GkJcE2vwGBLMO8m439lyRXpw4UL8RhhLNRVp47JNN8efbO1i28rn8GosZav6YJp/8B&lt;/diagram&gt;&lt;/mxfile&gt;">
<defs/>
<g>
<rect x="109" y="414" width="290" height="50" rx="7.5" ry="7.5" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<rect x="111" y="416" width="286" height="46" rx="6.9" ry="6.9" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 284px; height: 1px; padding-top: 439px; margin-left: 112px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
REST API
</div>
</div>
</div>
</foreignObject>
<text x="254" y="443" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
REST API
</text>
</switch>
</g>
<rect x="109" y="474" width="140" height="60" rx="9" ry="9" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<rect x="111" y="476" width="136" height="56" rx="8.4" ry="8.4" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 134px; height: 1px; padding-top: 504px; margin-left: 112px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
Authentication
<br/>
Management
</div>
</div>
</div>
</foreignObject>
<text x="179" y="508" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
Authentication...
</text>
</switch>
</g>
<rect x="259" y="474" width="140" height="60" rx="9" ry="9" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<rect x="261" y="476" width="136" height="56" rx="8.4" ry="8.4" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 134px; height: 1px; padding-top: 504px; margin-left: 262px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
FINDEX
<br/>
Server Side Engine
</div>
</div>
</div>
</foreignObject>
<text x="329" y="508" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
FINDEX...
</text>
</switch>
</g>
<rect x="109" y="80" width="290" height="50" rx="7.5" ry="7.5" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<rect x="111" y="82" width="286" height="46" rx="6.9" ry="6.9" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 284px; height: 1px; padding-top: 105px; margin-left: 112px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
Command Line User Interface
</div>
</div>
</div>
</foreignObject>
<text x="254" y="109" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
Command Line User Interface
</text>
</switch>
</g>
<rect x="109" y="130" width="290" height="100" rx="15" ry="15" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<rect x="111" y="132" width="286" height="96" rx="14.4" ry="14.4" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<path d="M 387.5 200 L 417.5 186 L 417.5 195.8 L 507.5 195.8 L 507.5 186 L 537.5 200 L 507.5 214 L 507.5 204.2 L 417.5 204.2 L 417.5 214 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/>
<rect x="129" y="180" width="123" height="40" rx="6" ry="6" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<rect x="131" y="182" width="119" height="36" rx="5.4" ry="5.4" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 117px; height: 1px; padding-top: 200px; margin-left: 132px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
FINDEX
<br/>
Client Side Engine
</div>
</div>
</div>
</foreignObject>
<text x="191" y="204" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
FINDEX...
</text>
</switch>
</g>
<rect x="179" y="122.5" width="150" height="60" fill="none" stroke="none" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 148px; height: 1px; padding-top: 153px; margin-left: 180px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center; max-height: 56px; overflow: hidden;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
Client Library
</div>
</div>
</div>
</foreignObject>
<text x="254" y="156" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
Client Library
</text>
</switch>
</g>
<rect x="264.5" y="180" width="123" height="40" rx="6" ry="6" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<rect x="266.5" y="182" width="119" height="36" rx="5.4" ry="5.4" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 117px; height: 1px; padding-top: 200px; margin-left: 268px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
KMS
<br/>
Connector
</div>
</div>
</div>
</foreignObject>
<text x="326" y="204" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
KMS...
</text>
</switch>
</g>
<path d="M 164 317.5 L 200 295 L 200 310.75 L 308 310.75 L 308 295 L 344 317.5 L 308 340 L 308 324.25 L 200 324.25 L 200 340 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" transform="rotate(90,254,317.5)" pointer-events="all"/>
<rect x="191.5" y="287.5" width="125" height="60" fill="none" stroke="none" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 123px; height: 1px; padding-top: 318px; margin-left: 193px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center; max-height: 56px; overflow: hidden;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
Encrypted Vectors
<br/>
and
<br/>
Encrypted Metadadata
</div>
</div>
</div>
</foreignObject>
<text x="254" y="321" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
Encrypted Vectors...
</text>
</switch>
</g>
<rect x="537.5" y="167.5" width="120" height="60" rx="9" ry="9" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-dasharray="12 12" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 198px; margin-left: 539px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
Key Management
<br/>
System
</div>
</div>
</div>
</foreignObject>
<text x="598" y="201" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
Key Management...
</text>
</switch>
</g>
<path d="M 206.5 645 C 206.5 636.72 227.77 630 254 630 C 266.6 630 278.68 631.58 287.59 634.39 C 296.5 637.21 301.5 641.02 301.5 645 L 301.5 695 C 301.5 703.28 280.23 710 254 710 C 227.77 710 206.5 703.28 206.5 695 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/>
<path d="M 301.5 645 C 301.5 653.28 280.23 660 254 660 C 227.77 660 206.5 653.28 206.5 645" fill="none" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 93px; height: 1px; padding-top: 683px; margin-left: 208px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
Key Value
<br/>
Store
</div>
</div>
</div>
</foreignObject>
<text x="254" y="686" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
Key Value...
</text>
</switch>
</g>
<rect x="109" y="539" width="290" height="50" rx="7.5" ry="7.5" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<rect x="111" y="541" width="286" height="46" rx="6.9" ry="6.9" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 284px; height: 1px; padding-top: 564px; margin-left: 112px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
Key Value Store Driver
</div>
</div>
</div>
</foreignObject>
<text x="254" y="568" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="12px" text-anchor="middle">
Key Value Store Driver
</text>
</switch>
</g>
<rect x="99" y="400" width="320" height="200" rx="30" ry="30" fill="none" stroke="#0066cc" pointer-events="all"/>
<rect x="0" y="486" width="80" height="36" fill="none" stroke="none" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 78px; height: 1px; padding-top: 504px; margin-left: 1px;">
<div data-drawio-colors="color: #0066CC; " style="box-sizing: border-box; font-size: 0px; text-align: center; max-height: 32px; overflow: hidden;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 102, 204); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
Server Side
<br/>
Applicatrion
</div>
</div>
</div>
</foreignObject>
<text x="40" y="508" fill="#0066CC" font-family="Helvetica" font-size="12px" text-anchor="middle">
Server Side...
</text>
</switch>
</g>
<rect x="94" y="67.5" width="320" height="170" rx="25.5" ry="25.5" fill="none" stroke="#0066cc" pointer-events="all"/>
<rect x="2" y="133" width="80" height="36" fill="none" stroke="none" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 78px; height: 1px; padding-top: 151px; margin-left: 3px;">
<div data-drawio-colors="color: #0066CC; " style="box-sizing: border-box; font-size: 0px; text-align: center; max-height: 32px; overflow: hidden;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 102, 204); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
Client Side
<br/>
Applicatrion
</div>
</div>
</div>
</foreignObject>
<text x="42" y="155" fill="#0066CC" font-family="Helvetica" font-size="12px" text-anchor="middle">
Client Side...
</text>
</switch>
</g>
<rect x="427.5" y="260" width="340" height="320" fill="none" stroke="none" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe flex-start; width: 338px; height: 1px; padding-top: 420px; margin-left: 430px;">
<div data-drawio-colors="color: #0066CC; " style="box-sizing: border-box; font-size: 0px; text-align: left; max-height: 316px; overflow: hidden;">
<div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 102, 204); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">
<ul>
<li>
The Fully Encrypted Vector Database is made of 2 components: one client side, one server side.
</li>
<li>
The client side component is the client library only or the Command Line Interface (wrapping the client library).
</li>
<li>
The server-side component is stateless and can be scaled by simple replication.
</li>
<li>
The server side maniuplates client side encrypted data and never decrypts them.
</li>
<li>
The KMS acts as a key management system and encryption oracle (keys never leave the KMS).
</li>
<li>
Encryption oif the vectors is performed client-side using Findex.
</li>
<li>
Encryption of the metadata is performed client-side, using Covercrypt, a post-quantum resistant algorithm with embedded access policies (hence providing data centric security).
</li>
</ul>
</div>
</div>
</div>
</foreignObject>
<text x="430" y="424" fill="#0066CC" font-family="Helvetica" font-size="12px">
The Fully Encrypted Vector Database is made of 2 compone...
</text>
</switch>
</g>
<rect x="9" y="0" width="100" height="40" fill="none" stroke="none" pointer-events="all"/>
<g transform="translate(-0.5 -0.5)">
<switch>
<foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;">
<div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe flex-start; width: 1px; height: 1px; padding-top: 20px; margin-left: 11px;">
<div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: left;">
<div style="display: inline-block; font-size: 24px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; font-weight: bold; white-space: nowrap;">
Lot 1.1: Fully Encrypted Vector Database
</div>
</div>
</div>
</foreignObject>
<text x="11" y="27" fill="rgb(0, 0, 0)" font-family="Helvetica" font-size="24px" font-weight="bold">
Lot 1.1:...
</text>
</switch>
</g>
</g>
<switch>
<g requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"/>
<a transform="translate(0,-5)" xlink:href="https://www.diagrams.net/doc/faq/svg-export-text-problems" target="_blank">
<text text-anchor="middle" font-size="10px" x="50%" y="100%">
Text is not SVG - cannot display
</text>
</a>
</switch>
</svg>
Side by side

After

Width:  |  Height:  |  Size: 27 KiB

19
documentation/docs/encrypting_and_decrypting_at_scale.md
View file

@ -1,5 +1,3 @@
# Encrypting and decrypting at scale
The Cosmian KMS is particularly suited for client-side encryption scenarios which may require
high-performance encryption and decryption.
@ -9,6 +7,23 @@ The KMS offers two mechanisms for encrypting and decrypting data:
parallelization, concurrency, and optimized batching capabilities.
- by using the `cosmian` CLI client to encrypt and decrypt data locally, including large files.
<!-- TOC -->
- [Calling the KMS API](#calling-the-kms-api)
- [Parallelization, concurrency, and batching](#parallelization-concurrency-and-batching)
- [Efficient batching](#efficient-batching)
- [The KMIP way](#the-kmip-way)
- [Optimized batching](#optimized-batching)
- [Encoding scheme](#encoding-scheme)
- [Performance heuristics](#performance-heuristics)
- [Using the Cosmian CLI client](#using-the-cosmian-cli-client)
- [Server side encryption and decryption](#server-side-encryption-and-decryption)
- [Available ciphers](#available-ciphers)
- [Format of the encrypted file](#format-of-the-encrypted-file)
- [Client side encryption and decryption](#client-side-encryption-and-decryption)
- [Available ciphers](#available-ciphers-1)
- [Format of the encrypted file](#format-of-the-encrypted-file-1)
<!-- TOC -->
## Calling the KMS API
The KMS provides a high-performance encryption and

184
documentation/docs/hsm.md
View file

@ -1,184 +0,0 @@
# HSM Support
The Cosmian KMS can be configured to use Proteccio HSMs to store and manage keys and create KMS keys wrapped by the HSM
keys. This provides the best of both worlds: the security of an HSM at rest and the scalability of a KMS at runtime.
Cosmian KMS natively integrates with
the [Proteccio](https://eviden.com/solutions/digital-security/data-encryption/trustway-proteccio-nethsm/) HSM.
## Main use case and benefit
The main use case for HSM support is to host keys in the KMS that are wrapped by keys stored in the HSM. This
combination provides the best of both worlds: the scalability and performance of the KMS at runtime, and the security of
the HSM at rest.
At rest, KMS keys are stored in the KMS database in a wrapped form, and the wrapping key is stored in the HSM. This
provides an additional layer of security for the keys stored in the KMS since the keys stored in the HSM are protected
by the HSM's hardware security mechanisms, and benefit from the HSM certifications.
At runtime, however, encryption and decryption requests from applications are processed by the KMS, which first unwraps
the keys stored in the KMS database using the keys stored in the HSM. Contrarily to the HSM, the KMS is a highly
scalable and performant system that can handle a large number of requests concurrently.
## Setup
This solution works on Linux (x64_86) and has been validated against the Proteccio `nethsm` library version 3.17.
The KMS expects the Proteccio `nethsm` library to be installed in `/lib/libnethsm.so` and the Proteccio configuration
files in `/etc/proteccio`. Please run the `nethsmstatus` tool to check the status of the HSM before proceeding with the
rest of the installation.
The KMS command line arguments to enable HSM support are:
```shell
--hsm-model "proteccio" \
--hsm-admin "<HSM_ADMIN_USERNAME>" \
--hsm-slot <number_of_slot1> --hsm-password <password_of_slot1> \
--hsm-slot <number_of_slot2> --hsm-password <password_of_slot2>
...
```
The `--hsm-model` argument is the HSM model to be used; only `proteccio` is supported in this release.
The `--hsm-admin` argument is the username of the HSM administrator. The HSM administrator is the only user that can
create objects on the HSM via the KMIP `Create` operation the delegate other operations to other users. (see below)
The `--hsm-slot` and `--hsm-password` arguments are the slot number and password of the HSM slots to be used by the KMS.
These arguments can be repeated multiple times to specify multiple slots.
If using the TOML configuration file, see this [page](./server_configuration_file.md#toml-configuration-file) for more information on how to
configure the HSM support.
## HSM operations
HSM keys are created with a unique identifier that is pre-fixed by the `hsm` keyword and the slot number in the form:
```shell
hsm::<slot_number>::<key_identifier>
```
For instance, the key `hsm::1::mykey` is a key stored in the HSM slot 1 with the identifier `mykey`. Technically, this
identifier is stored in the `LABEL` field of the key object in the HSM.
The following KMIP operations can be performed on HSM keys via the KMS server API:
### `Create`
Create a new key in the HSM. The key unique must be provided on the request and must follow the
`hsm::<slot_number>::<key_identifier>` format described above.
Only the user identified by the `--hsm-admin` argument can create keys in the HSM.
RSA and AES keys are supported.
When creating an RSA key, the `key_identifier` will be that of the private key. The corresponding public key will be
automatically created and stored in the HSM with the same `key_identifier` but with the `_pk` suffix, for example, the
public key of the `hsm::1::mykey` private key will be created with unique identifier `has::1::mykey_pk`.
Using the `ckms` client, an RSA 4096-bit key can be created with the following command:
```shell
❯ ckms rsa keys create --size_in_bits 4096 --sensitive hsm::4::mykey
The RSA key pair has been created.
Public key unique identifier: hsm::4::mykey_pk
Private key unique identifier: hsm::4::mykey
```
Keys should be flagged as `sensitive` when created in the HSM, so that the private key or symmetric key cannot be
exported (see below `Get` and `Export`).
Note: HSM keys do not support object tagging in this release.
### `Destroy`
Contrarily to the KMS keys, HSM keys must not be Revoked before being Destroyed. The `Destroy` operation will remove the
key from the HSM.
Only the user identified by the `--hsm-admin` argument or a user which has been granted the `Destroy` operation (by the
HSM admin) can destroy keys in the HSM.
To destroy the key `hsm::4::mykey`, the following command can be used:
```shell
❯ ckms rsa keys destroy --key-id hsm::4::mykey
Successfully destroyed the key.
Unique identifier: hsm::4::mykey
```
### `Get` & `Export`
The `Get` and `Export` operations are used to retrieve the key material from the HSM.
Only the user identified by the `--hsm-admin` argument or a user which has been granted the `Get` operation (by the HSM
admin) can retrieve keys from the HSM.
Private keys or symmetric keys marked as `sensitive` cannot be retrieved from the HSM. The public key of a keypair can
always be retrieved.
To export the public key `hsm::4::mykey_pk` in PKCS#8 PEM format, the following command can be used:
```shell
❯ ckms rsa keys export --key-id hsm::4::mykey_pk --key-format pkcs8-pem /tmp/pubkey.pem
The key hsm::4::mykey_pk of type PublicKey was exported to "/tmp/pubkey.pem"
Unique identifier: hsm::4::mykey_pk
```
### `Encrypt`
Symmetric keys and public keys can be used to encrypt data. Only the user identified by the `--hsm-admin` argument or a
user which has been granted the `Encrypt` operation (by the HSM admin) can encrypt data with keys stored in the HSM.
For symmetric keys, only AES GCM is supported. For RSA keys, CKM_RSA_PKCS_OAEP and the now deprecated, but still widely
used, CKM_RSA_PKCS (v1.5) are supported. The hashing algorithm is fixed to SHA256.
To encrypt a message with the public key `hsm::4::mykey_pk` and the CKM RSA PKCS OAEP algorithm, the following command
can be used:
```shell
❯ ckms rsa encrypt --key-id hsm::4::mykey_pk --encryption-algorithm ckm-rsa-pkcs-oaep \
/tmp/secret.pem
The encrypted file is available at "/tmp/secret.enc"
```
### `Decrypt`
Symmetric keys and private keys can be used to decrypt data. Only the user identified by the `--hsm-admin` argument or a
user which has been granted the `Decrypt` operation (by the HSM admin) can decrypt data with keys stored in the HSM.
For symmetric keys, only AES GCM is supported. For RSA keys, CKM_RSA_PKCS_OAEP and the now deprecated, but still widely
used, CKM_RSA_PKCS (v1.5) are supported. The hashing algorithm is fixed to SHA256.
To decrypt a message with the private
key `hsm::4::mykey` and the CKM RSA PKCS OAEP algorithm, the following command can be used:
```shell
❯ ckms rsa decrypt --key-id hsm::4::mykey --encryption-algorithm ckm-rsa-pkcs-oaep \
/tmp/secret.enc
The decrypted file is available at "/tmp/secret.plain"
```
## Creating a KMS key wrapped by an HSM key
To create a KMS key wrapped by an HSM key, the `--wrapping-key-id` argument must be used to specify the unique
identifier of the HSM key.
The user creating the key must be the HSM admin (see above) or have been granted the `Encrypt` operation on the HSM key.
For instance, the following command creates a 256-bit AES key wrapped by the HSM RSA (public) key `hsm::4::mykey_pk`:
```shell
❯ ckms sym keys create --algorithm aes --number-of-bits 256 --sensitive \
--wrapping-key-id hsm::4::mykey_pk my_sym_key
The symmetric key was successfully generated.
Unique identifier: my_sym_key
```
The symmetric key is now stored in the database encrypted (wrapped) by the HSM key. The encryption happened in the HSM.
The key can now be used to encrypt, and decrypt data, and the KMS will transparently unwrap the key using the HSM key.
This unwrapping will happen once, and the unwrapped symmetric key will be cached in memory for later operations; no
clear text symmetric key will be stored in the KMS database.
For example, to encrypt a message with the key `my_sym_key`, the following command can be used:
```shell
❯ ckms sym encrypt --key-id my_sym_key /tmp/secret.txt
The encrypted file is available at "/tmp/secret.enc"
```

246
documentation/docs/hsms/hsm_operations.md Normal file
View file

@ -0,0 +1,246 @@
In addition to managing its own keys, Cosmian KMS can act as a proxy to an HSM to store and manage keys in the HSM.
<!-- TOC -->
- [HSM keys](#hsm-keys)
- [KMIP operations](#kmip-operations)
- [Create](#create)
- [Destroy](#destroy)
- [Get \& Export](#get--export)
- [Encrypt](#encrypt)
- [Decrypt](#decrypt)
- [Creating a KMS key wrapped by an HSM key](#creating-a-kms-key-wrapped-by-an-hsm-key)
- [Small data: encrypting server side](#small-data-encrypting-server-side)
- [Large data: encrypting client side with key wrapping](#large-data-encrypting-client-side-with-key-wrapping)
<!-- TOC -->
## HSM keys
HSM keys are prefixed keys. They are created with a unique identifier that is pre-fixed by the `hsm` keyword and the
slot
number in the form:
```shell
hsm::<slot_number>::<key_identifier>
```
For instance, the key `hsm::1::mykey` is a key stored in the HSM slot 1 with the identifier `mykey`. Technically, this
identifier is stored in the `LABEL` field of the key object in the HSM.
Non-prefixed keys are considered KMS keys and are stored in the KMS database.
## KMIP operations
Some KMIP operations can be performed on HSM keys via the KMS server API.
First, make sure the HSM is configured and that the [Cosmian CLI](https://package.cosmian.com/cli/) is installed and
configured.
### Create
Create a new key in the HSM. The key unique must be provided on the request and must follow the
`hsm::<slot_number>::<key_identifier>` format described above.
Only the user identified by the `--hsm-admin` argument can create keys in the HSM.
RSA and AES keys are supported.
When creating an RSA key, the `key_identifier` will be that of the private key. The corresponding public key will be
automatically created and stored in the HSM with the same `key_identifier` but with the `_pk` suffix, for example, the
public key of the `hsm::1::mykey` private key will be created with unique identifier `has::1::mykey_pk`.
Create an RSA 4096-bit key with the Cosmiian CLI:
```shell
❯ cosmian kms rsa keys create --size_in_bits 4096 --sensitive hsm::4::my_rsa_key
The RSA key pair has been created.
Public key unique identifier: hsm::4::my_rsa_key_pk
Private key unique identifier: hsm::4::my_rsa_key
```
Create an AES 256-bit key with the Cosmiian CLI:
```shell
❯ cosmian kms sym keys create --algorithm aes --number-of-bits 256 --sensitive hsm::4::my_aes_key
The symmetric key was successfully generated.
Unique identifier: hsm::4::my_aes_key
```
Keys should be flagged as `sensitive` when created in the HSM, so that the private key or symmetric key cannot be
exported (see below `Get` and `Export`).
Note: HSM keys do not support object tagging in this release.
### Destroy
Contrarily to the KMS keys, HSM keys must not be Revoked before being Destroyed. The `Destroy` operation will remove the
key from the HSM.
Only the user identified by the `--hsm-admin` argument or a user which has been granted the `Destroy` operation (by the
HSM admin) can destroy keys in the HSM.
To destroy the key `hsm::4::my_rsa_key`, the following command can be used:
```shell
❯ cosmian kms rsa keys destroy --key-id hsm::4::my_rsa_key
Successfully destroyed the key.
Unique identifier: hsm::4::mykey
```
To destroy the corresponding public key `hsm::4::my_rsa_key_pk`, the following command can be used:
```shell
❯ cosmian kms rsa keys destroy --key-id hsm::4::my_rsa_key_pk
Successfully destroyed the object.
Unique identifier: hsm::4::my_rsa_key_pk
```
### Get & Export
The `Get` and `Export` operations are used to retrieve the key material from the HSM.
Only the user identified by the `--hsm-admin` argument or a user which has been granted the `Get` operation (by the HSM
admin) can retrieve keys from the HSM.
Private keys or symmetric keys marked as `sensitive` cannot be retrieved from the HSM. The public key of a keypair can
always be retrieved.
To export the public key `hsm::4::my_rsa_key_pk` in PKCS#8 PEM format, the following command can be used:
```shell
❯ cosmian kms rsa keys export --key-id hsm::4::my_rsa_key_pk --key-format pkcs8-pem /tmp/pubkey.pem
The key hsm::4::my_rsa_key_pk of type PublicKey was exported to "/tmp/pubkey.pem"
Unique identifier: hsm::4::my_rsa_key_pk
```
To export the private key `hsm::4::mykey` in PKCS#8 PEM format, the following command can be used:
```shell
❯ cosmian kms rsa keys export --key-id hsm::4::my_rsa_key --key-format pkcs8-pem /tmp/privkey.pem
The key hsm::4::my_rsa_key of type PrivateKey was exported to "/tmp/privkey.pem"
Unique identifier: hsm::4::my_rsa_key
```
To export the symmetric key `hsm::4::my_aes_key` in raw format (i.e. raw bytes),
the following command can be used:
```shell
❯ cosmian kms sym keys export --key-id hsm::4::my_aes_key --key-format raw /tmp/symkey.raw
The key hsm::4::my_aes_key of type SymmetricKey was exported to "/tmp/symkey.raw"
Unique identifier: hsm::4::my_aes_key
```
### Encrypt
Symmetric keys and public keys can be used to encrypt data. Only the user identified by the `--hsm-admin` argument or a
user which has been granted the `Encrypt` operation (by the HSM admin) can encrypt data with keys stored in the HSM.
For symmetric keys, only AES GCM is supported. For RSA keys, CKM_RSA_PKCS_OAEP and the now deprecated, but still widely
used, CKM_RSA_PKCS (v1.5) are supported. The hashing algorithm is fixed to SHA256.
When using RSA the maximum message size in bytes is:
- PKCS#1 v1.5: (key size in bits / 8) - 11
- OAEP: (key size in bits / 8) - 66
To encrypt a message with the public key `hsm::4::my_rsa_key_pk` and the CKM RSA PKCS OAEP algorithm,
the following command can be used:
```shell
❯ cosmian kms rsa encrypt --key-id hsm::4::my_rsa_key_pk --encryption-algorithm ckm-rsa-pkcs-oaep \
/tmp/secret.txt
The encrypted file is available at "/tmp/secret.enc"
```
To encrypt a message using AES GCM with the symmetric key `hsm::4::my_aes_key`, the following command can be used:
```shell
❯ cosmian kms sym encrypt --key-id hsm::4::my_aes_key --data-encryption-algorithm aes-gcm /tmp/secret.txt
The encrypted file is available at "/tmp/secret.enc"
```
### Decrypt
Symmetric keys and private keys can be used to decrypt data. Only the user identified by the `--hsm-admin` argument or a
user which has been granted the `Decrypt` operation (by the HSM admin) can decrypt data with keys stored in the HSM.
For symmetric keys, only AES GCM is supported. For RSA keys, CKM_RSA_PKCS_OAEP and the now deprecated, but still widely
used, CKM_RSA_PKCS (v1.5) are supported. The hashing algorithm is fixed to SHA256.
To decrypt a message with the private
key `hsm::4::hsm::4::my_rsa_key` and the CKM RSA PKCS OAEP algorithm, the following command can be used:
```shell
❯ cosmian kms rsa decrypt --key-id hsm::4::my_rsa_key --encryption-algorithm ckm-rsa-pkcs-oaep \
--output-file /tmp/secret.recovered.txt /tmp/secret.enc
The decrypted file is available at "/tmp/secret.plain"
```
To decrypt a message using AES GCM with the symmetric key `hsm::4::my_aes_key`, the following command can be used:
```shell
> cosmian kms sym decrypt --key-id hsm::4::my_aes_key --data-encryption-algorithm AesGcm \
--output-file /tmp/secret.recovered.txt /tmp/secret.enc
The decrypted file is available at "/tmp/secret.recoverd.txt"
```
## Creating a KMS key wrapped by an HSM key
To create a KMS key wrapped by an HSM key, the `--wrapping-key-id` argument must be used to specify the unique
identifier of the HSM key.
The user creating the key must be the HSM admin (see above) or have been granted the `Encrypt` operation on the HSM key.
For instance, the following command creates a 256-bit AES key wrapped by the HSM RSA (public) key
`hsm::4::my_rsa_key_pk`:
```shell
> cosmian kms sym keys create --algorithm aes --number-of-bits 256 --sensitive \
--wrapping-key-id hsm::4::my_rsa_key_pk my_sym_key
The symmetric key was successfully generated.
Unique identifier: my_sym_key
```
The symmetric key is now stored in the database encrypted (wrapped) by the HSM key. The encryption happened in the HSM.
The symmetric key can now be used to encrypt, and decrypt data, and the KMS will transparently unwrap the key using the
HSM key.
This unwrapping will happen once, and the unwrapped symmetric key will be cached in memory for later operations; no
clear text symmetric key will be stored in the KMS database.
### Small data: encrypting server side
For example, to encrypt a message with the key `my_sym_key` server side, the following command can be used:
```shell
> cosmian kms sym encrypt --key-id my_sym_key /tmp/secret.txt
The encrypted file is available at "/tmp/secret.enc"
```
To decrypt a message with the key `my_sym_key`, the following command can be used:
```shell
> cosmian kms sym decrypt --key-id my_sym_key --output-file /tmp/secret.recovered.txt /tmp/secret.enc
The decrypted file is available at "/tmp/secret.recovered.txt"
```
#### Large data: encrypting client side with key wrapping
To encrypt a large file with the key `my_sym_key` client side, the following command can be used:
```shell
>cosmian kms sym encrypt --key-id my_sym_key_2 --data-encryption-algorithm aes-gcm \
--key-encryption-algorithm rfc5649 /tmp/large.bin
The encrypted file is available at "/tmp/large.enc"
```
In this case an ephemeral symmetric key (the Data Encryption Key, DEK)
is generated and used to encrypt the data. The DEK is then encrypted/wrapped with RFC4659 (a.k.a NIST AES Key Wrap)
with the key `my_sym_key`, called the Key Encryption Key, KEK. The wrapping of the DEK by the KEK
is stored at the beginning of the encrypted file.
At rest, in the KMS database, `my_sym_key` is stored encrypted/wrapped with the HSM key `hsm::4::my_rsa_key_pk`.
To decrypt a large file with the KEK `my_sym_key` client side, the following command can be used:
```shell
> cosmian kms sym decrypt --key-id my_sym_key_2 --data-encryption-algorithm aes-gcm \
--key-encryption-algorithm rfc5649 --output-file /tmp/large.recoverd.bin /tmp/large.enc
The decrypted file is available at "/tmp/large.recoverd.bin"
```

44
documentation/docs/hsms/index.md Normal file
View file

@ -0,0 +1,44 @@
# HSM Support
The Cosmian KMS can be configured to use HSMs to store and manage keys and create KMS keys
wrapped by the HSM
keys. This provides the best of both worlds: the security of an HSM at rest and the scalability of a KMS at runtime.
Cosmian KMS natively integrates with
the [Proteccio](https://eviden.com/solutions/digital-security/data-encryption/trustway-proteccio-nethsm/) and
the [Utimaco general purpose](https://utimaco.com/solutions/applications/general-purpose-hardware-security-modules)
HSMs.
## Main use case and benefits
Aside from providing a single interface to manage both KMS and HSM keys,
the main use case for HSM support is to host keys in the KMS that
are [wrapped by keys stored in the HSM](./hsm_operations.md/#creating-a-kms-key-wrapped-by-an-hsm-key).
This combination provides the best of both worlds:
- the **scalability** and performance of the KMS **at runtime** to answer a large number of requests concurrently,
- and the **hardware security** of the HSM **at rest**, which may be a compliance requirement in some industries.
Typical use cases include:
- securing workplace applications such as [MS 365](https://www.microsoft.com/en-us/microsoft-365)
or [Google Workspace](https://workspace.google.com),
where concurrent requests from
potentially a large
number of users need to be processed quickly,
- securing big data applications such as
Hadoop/Spark, [Snowflake](https://snowflake.com), [Databricks](https://databricks.com) where a large number of
encryption and decryption requests need to be processed on each request on the fly.
### At Rest
KMS keys are stored in the KMS database in a wrapped form, and the wrapping key is stored in the HSM. This
provides an additional layer of security for the keys stored in the KMS since the keys stored in the HSM are protected
by the HSM's hardware security mechanisms, and benefit from the HSM certifications.
#### At Runtime
Encryption and decryption requests from applications are processed by the KMS, which first unwraps
the keys stored in the KMS database using the keys stored in the HSM. Contrarily to the HSM, the KMS is a highly
scalable and performant system that can handle a large number of requests concurrently.

45
documentation/docs/hsms/proteccio.md Normal file
View file

@ -0,0 +1,45 @@
Cosmian KMS natively integrates with
the [Trustway Proteccio](https://eviden.com/solutions/digital-security/data-encryption/trustway-proteccio-nethsm/) HSM.
### Proteccio library setup
This solution works on Linux (x86_64) and has been validated against the Proteccio `nethsm` library version 3.17.
The KMS expects:
- the Proteccio `nethsm` library to be installed in `/lib/libnethsm.so`
- and the Proteccio configuration files in `/etc/proteccio`.
Please run the `nethsmstatus` tool to check the status of the HSM before proceeding with the
rest of the installation.
### KMS configuration
At least one slot and its corresponding password must be configured. Any slot and any number of slots may be used.
When using the [TOML configuration file](../server_configuration_file.md#toml-configuration-file), the HSM support
is enabled by configuring these 4 parameters:
```toml
hsm_model = "proteccio"
hsm_admin = "<HSM_ADMIN_USERNAME>" # defaults to "admin"
hsm_slot = [0, 0, ] # example [1,4] for slots 1 and 4
hsm_password = ["<password>", "<password>", ] # example ["pass1", "pass4"] for slots 1 and 4
```
Even if only one slot is used, the `hsm_slot` and `hsm_password` parameters must be arrays.
When the KMS is started from the command line, the HSM support can be enabled by using the following arguments:
```shell
--hsm-model "proteccio" \
--hsm-admin "<HSM_ADMIN_USERNAME>" \
--hsm-slot <number_of_1st_slot> --hsm-password <password_of_1st_slot> \
--hsm-slot <number_of_2nd_slot> --hsm-password <password_of_2nd_slot>
```
The `hsm-model` parameter is the HSM model to be used; use `proteccio`
The `hsm-admin` parameter is the username of the HSM administrator. The HSM administrator is the only user that can create objects on the HSM via the KMIP `Create` operation the delegate other operations to other users. (see below)
The `hsm-slot` and `hsm-password` parameters are the slot number and password of the HSM slots to be used by the KMS. These arguments can be repeated multiple times to specify multiple slots.

112
documentation/docs/hsms/utimaco.md Normal file
View file

@ -0,0 +1,112 @@
This solution works on Linux (x86_64) and has been validated against the Utimaco client library version 6.0.
### Utimaco library setup
This solution works on Linux (x64_86) and has been validated against the Utimaco `libcs_pkcs11_R3.so` library version 6.0.
The KMS expects:
- the Utimaco `cs_pkcs11_R3` library to be installed in `/lib/libcs_pkcs11_R3.so`
- the Utimaco configuration file `cs_pkcs11_R3.cfg` to be in `/etc/utimaco` and
- and the environment variable `CS_PKCS11_R3_CF` to point to it, i.e.,
```sh
export CS_PKCS11_R3_CFG=/etc/utimaco/cs_pkcs11_R3.cfg
```
Please make sure the `cs_pkcs11_R3.cfg` is set with the correct parameter, and validate your
installation with the `p11tool2` utility, by running, for instance,
```sh
./p11tool2 Slot=0 GetSlotInfo
```
### KMS configuration
At least one slot and its corresponding password must be configured. Any slot and any number of slots may be used.
When using the [TOML configuration file](../server_configuration_file.md#toml-configuration-file), the HSM support is enabled by configuring these 4 parameters:
```toml
hsm_model = "utimaco"
hsm_admin = "<HSM_ADMIN_USERNAME>" # defaults to "admin"
hsm_slot = [0, 0, ] # example [0,4] for slots 0 and 4
hsm_password = ["<password>", "<password>", ] # example ["pass0", "pass4"] for slots 0 and 4
```
Even if only one slot is used, the `hsm_slot` and `hsm_password` parameters must be arrays.
When the KMS is started from the command line, the HSM support can be enabled by using the following arguments:
```shell
--hsm-model "utimaco" \
--hsm-admin "<HSM_ADMIN_USERNAME>" \
--hsm-slot <number_of_1st_slot> --hsm-password <password_of_1st_slot> \
--hsm-slot <number_of_2and_slot> --hsm-password <password_of_2and_slot>
```
The `hsm-model` parameter is the HSM model to be used; use `utimaco`.
The `hsm-admin` parameter is the username of the HSM administrator. The HSM administrator is the only user that can create objects on the HSM via the KMIP `Create` operation the delegate other operations to other users. (see below)
The `hsm-slot` and `hsm-password` parameters are the slot number and user password of the HSM slots to be used by the KMS. These arguments can be repeated multiple times to specify multiple slots.
### Using the simulator
Utimaco provides a simulator that can be used in lieu of a physical HSM to test your installation.
The simulator is a 32-bit Linux i386 library (it also exists as a Windows binary).
To install the simulator on a Debian based (e.g. Ubuntu) Linux amd64/x86_64, follow these general steps.
1. Enable 32 bit support
```bash
sudo dpkg --add-architecture i386
```
Then
```bash
sudo apt-get update
sudo apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386
```
2. Start the simulator
In `<eval-bundle-6.0.0>\Software\Windows\Simulator\sim5_windows\bin`, run
```sh
.\bl_sim5.exe -h -o -d ..\devices\
```
3. Make sure the Device in `cs_pkcs11_R3.cfg` points to the simulator.
4. Initialize a slot and create the Security Officer and User pins.
Due to a bug (?) in the simulator, the Security Officer PIN must be set **then changed** before the User PIN can be set, and **then changed** as well.
```bash
# Set the SO PIN to 11223344
./p11tool2 Slot=0 login=ADMIN,./key/ADMIN_SIM.key InitToken=11223344
# Change the SO PIN to 12345678
./p11tool2 Slot=0 LoginSO=11223344 SetPin=11223344,12345678
```
Failing to change the SO PIN before setting the User PIN will result in the following error: `Error 0x000001B8 (
CKR_PIN_TOO_WEAK)`
```bash
# Set the User PIN to 11223344
./p11tool2 Slot=0 LoginSO=12345678 InitPin=11223344
# Change the User PIN to 12345678
./p11tool2 Slot=0 LoginUser=11223344 SetPin=11223344,12345678
```
Now, both the SO and User PINs have been set to 12345678.
To list objects on Slot 0, use:
```bash
./p11tool2 Slot=0 LoginUser=12345678 ListObjects
```

76
documentation/docs/index.md
View file

@ -1,5 +1,3 @@
# Getting started
The **Cosmian KMS** is a high-performance,
[**open-source**](https://github.com/Cosmian/kms),
[FIPS 140-3 compliant](./fips.md) server application
@ -7,87 +5,61 @@ written in [**Rust**](https://www.rust-lang.org/) that presents some unique feat
- the ability to confidentially run in a public cloud — or any zero-trust environment — using
Cosmian VM. See our cloud-ready confidential KMS on the
[Azure, GCP, and AWS marketplaces](https://cosmian.com/marketplaces/) and our [deployment guide](./marketplace_guide.md)
[Azure, GCP, and AWS marketplaces](https://cosmian.com/marketplaces/) and
our [deployment guide](installation/marketplace_guide.md)
- support of state-of-the-art authentication mechanisms (see [authentication](./authentication.md))
- out-of-the-box support of
[Google Workspace Client Side Encryption (CSE)](./google_cse/index.md)
- out-of-the-box support
of [Microsoft Double Key Encryption (DKE)](./ms_dke/index.md)
- support for the [Proteccio HSM](./hsm.md) with KMS keys wrapped by the HSM
- support for [HSMs](./hsms/index.md) (trustway Proteccio, Utimaco general purpose) with KMS keys wrapped by the HSM
- [Veracrypt](./pkcs11/veracrypt.md)
and [LUKS](./pkcs11/luks.md) disk encryption support
- [FIPS 140-3](./fips.md) mode gated behind the feature `fips`
- a [JSON KMIP 2.1](./kmip_2_1/index.md) compliant interface
- a full-featured client [command line and graphical interface](../cosmian_cli/index.md)
- a [high-availability mode](./high_availability_mode.md) with simple horizontal scaling
- a [high-availability mode](installation/high_availability_mode.md) with simple horizontal scaling
- a support of Python, Javascript, Dart, Rust, C/C++, and Java clients (see the `cloudproof` libraries
on [Cosmian Github](https://github.com/Cosmian))
- integrated with [OpenTelemetry](https://opentelemetry.io/)
The **Cosmian KMS** is both a Key Management System and a Public Key Infrastructure.
As a KMS, it is designed to manage the lifecycle of keys and provide scalable cryptographic
services such as on-the-fly key generation, encryption, and decryption operations.
The **Cosmian KMS** is a Key Management System, an Encryption Oracle and a Public Key Infrastructure.
- As **a key management system**, it is designed to manage the lifecycle of keys and provide services such as on-the-fly
key generation and revocation, including in [connected HSMs](./hsms/index.md).
- As en **encryption oracle**, it provides high-availability, high-scalability, encryption, and decryption operations.
This
is the Cosmian KMS strong point, offering **millions of operations in seconds** while providing high security for keys
when [backed by an HSM](./hsms/index.md).
- As a **PKI** it can manage root and intermediate certificates, sign and verify certificates, use
their public keys to encrypt and decrypt data.
Certificates can be exported under various formats including _PKCS#12_ modern and legacy flavor,
to be used in various applications, such as in _S/MIME_ encrypted emails.
The **Cosmian KMS** supports all the standard NIST cryptographic algorithms as well as advanced post-quantum
cryptography algorithms such as [Covercrypt](https://github.com/Cosmian/cover_crypt).
Please refer to the list of [supported algorithms](./algorithms.md).
As a **PKI** it can manage root and intermediate certificates, sign and verify certificates, use
their public keys to encrypt and decrypt data.
Certificates can be exported under various formats including _PKCS#12_ modern and legacy flavor,
to be used in various applications, such as in _S/MIME_ encrypted emails.
## Easy to deploy
The **Cosmian KMS** is packaged as:
- [Debian](https://package.cosmian.com/kms/4.21.2/ubuntu-22.04/) or [RPM](https://package.cosmian.com/kms/4.21.2/rhel9/) package
- Docker [image](https://github.com/Cosmian/kms/pkgs/container/kms) and [FIPS image](https://github.com/Cosmian/kms/pkgs/container/kms)
- [Debian](https://package.cosmian.com/kms/4.21.2/ubuntu-22.04/) or [RPM](https://package.cosmian.com/kms/4.21.2/rhel9/)
package
- Docker [image](https://github.com/Cosmian/kms/pkgs/container/kms)
and [FIPS image](https://github.com/Cosmian/kms/pkgs/container/kms)
- Pre-built [binaries](https://package.cosmian.com/kms/4.21.2/) for multiple operating systems (Linux, Windows, MacOS)
## Client CLI
The **Cosmian KMS** has an easy-to-use client command line interface built for many operating systems.
The [Cosmian CLI](../cosmian_cli/index.md) can manage the server, and the keys and perform operations such as encryption or decryption.
The [Cosmian CLI](../cosmian_cli/index.md) can manage the server, and the keys and perform operations such as encryption
or decryption.
The **[Cosmian CLI](../cosmian_cli/index.md)** is packaged as:
- [Debian](https://package.cosmian.com/kms/4.21.2/ubuntu-22.04/) or [RPM](https://package.cosmian.com/kms/4.21.2/rhel9/) package
- [Debian](https://package.cosmian.com/kms/4.21.2/ubuntu-22.04/) or [RPM](https://package.cosmian.com/kms/4.21.2/rhel9/)
package
- Pre-built [binaries](https://package.cosmian.com/cli/) for multiple operating systems (Linux, Windows, MacOS)
**Note:** `ckms` has been replaced by [Cosmian CLI](../cosmian_cli/index.md) to manage other Cosmian products.
!!! info "Quick start"
To quick-start a Cosmian KMS server on `http://localhost:9998` that stores its data
inside the container, simply run the following command:
```sh
docker run -p 9998:9998 --name kms ghcr.io/cosmian/kms:latest
```
Using [Cosmian CLI](../cosmian_cli/index.md), you can easily manage the server:
1) Create a 256-bit symmetric key
```sh
cosmian kms sym keys create --number-of-bits 256 --algorithm aes --tag my-file-key
...
The symmetric key was successfully generated.
Unique identifier: 87e9e2a8-4538-4701-aa8c-e3af94e44a9e
```
2) Encrypt the `image.png` file with AES GCM using the key
```sh
cosmian kms sym encrypt --tag my-file-key --output-file image.enc image.png
...
The encrypted file is available at "image.enc"
```
3) Decrypt the `image.enc` file using the key
```sh
cosmian kms sym decrypt --tag my-file-key --output-file image2.png image.enc
...
The decrypted file is available at "image2.png"
```

11
documentation/docs/high_availability_mode.md → documentation/docs/installation/high_availability_mode.md
View file

@ -1,14 +1,17 @@
This mode offers high availability through redundancy and load-balancing.
The KMS servers are stateless, so they can simply be scaled horizontally by connecting them to the same database and fronting them with a load balancer.
The KMS servers are stateless, so they can simply be scaled horizontally by connecting them to the same database and
fronting them with a load balancer.
![high-availability](./drawings/high-availability.drawio.svg)
![high-availability](../drawings/high-availability.drawio.svg)
### Configuring the load balancer
Since the KMS servers are stateless, any load-balancing strategy may be selected, such as a simple round-robin.
When the Cosmian KMS servers are configured to export an HTTPS port (as is the case when running inside a confidential VM):
When the Cosmian KMS servers are configured to export an HTTPS port (as is the case when running inside a confidential
VM):
- all the Cosmian KMS servers should expose the same server certificate on their HTTPS port
- and the load balancer should be configured as an SSL load balancer (HAProxy is a good example of a high-performance SSL load balancer)
- and the load balancer should be configured as an SSL load balancer (HAProxy is a good example of a high-performance
SSL load balancer)

178
documentation/docs/installation/installation_getting_started.md Normal file
View file

@ -0,0 +1,178 @@
Cosmian KMS may be installed on a variety of platforms, including Docker, Ubuntu, RHEL, MacOS, and Windows.
It is also available on the major cloud providers marketplaces, prepackaged to run confidentially in a Cosmian VM.
Please check [this page](./marketplace_guide.md) for more information.
When installed using the options below, the KMS server will be automatically configured to run
using an SQLite database.
If you wish to change the database configuration, please refer to the [database guide](../database.md).
For high availability and scalability, please refer to the [high availability guide](./high_availability_mode.md).
!!!info "Cosmian CLI"
The Cosmian CLI lets you interact with the KMS from the command line.
Install it from [Cosmian CLI](https://package.cosmian.com/cli/)
and [configure it](../../cosmian_cli/index.md).
=== "Docker"
Run the container as follows:
```sh
docker run -p 9998:9998 --name kms ghcr.io/cosmian/kms:latest
```
The KMS will be available on `http://localhost:9998`, and the server will store its data inside the
container in the `/root/cosmian-kms/sqlite-data` directory.
FIPS version is also available:
```sh
docker run -p 9998:9998 --name kms ghcr.io/cosmian/kms-fips:latest
```
To persist data between restarts, map the `/root/cosmian-kms/sqlite-data` path to a filesystem
directory or a Docker volume, e.g. with a volume named `cosmian-kms`:
```sh
docker run --rm -p 9998:9998 \
-v cosmian-kms:/root/cosmian-kms/sqlite-data \
--name kms ghcr.io/cosmian/kms:latest
```
=== "Ubuntu 20.04"
Download package and install it:
```sh
sudo apt update && sudo apt install -y wget
wget https://package.cosmian.com/kms/4.21.2/ubuntu-20.04/cosmian-kms-server_4.21.0-1_amd64.deb
sudo apt install ./cosmian-kms-server_4.21.0-1_amd64.deb
cosmian_kms_server --version
```
Or install the FIPS version:
```sh
wget https://package.cosmian.com/kms/4.21.2/ubuntu-20.04/cosmian-kms-server-fips_4.21.0-1_amd64.deb
sudo apt install ./cosmian-kms-server-fips_4.21.0-1_amd64.deb
cosmian_kms_server --version
```
A `cosmian_kms` service will be configured; the service file is located at `/etc/systemd/system/cosmian_kms.service`.
The server will use a configuration file located at `/etc/cosmian_kms/kms.toml`.
To start the KMS, run:
```sh
sudo systemctl start cosmian_kms
```
=== "Ubuntu 22.04"
Download package and install it:
```sh
sudo apt update && sudo apt install -y wget
wget https://package.cosmian.com/kms/4.21.2/ubuntu-22.04/cosmian-kms-server_4.21.0-1_amd64.deb
sudo apt install ./cosmian-kms-server_4.21.0-1_amd64.deb
cosmian_kms_server --version
```
Or install the FIPS version:
```sh
wget https://package.cosmian.com/kms/4.21.2/ubuntu-22.04/cosmian-kms-server-fips_4.21.0-1_amd64.deb
sudo apt install ./cosmian-kms-server-fips_4.21.0-1_amd64.deb
cosmian_kms_server --version
```
A `cosmian_kms` service will be configured; the service file is located at `/etc/systemd/system/cosmian_kms.service`.
The server will use a configuration file located at `/etc/cosmian_kms/kms.toml`.
To start the KMS, run:
```sh
sudo systemctl start cosmian_kms
```
=== "Ubuntu 24.04"
Download package and install it:
```sh
sudo apt update && sudo apt install -y wget
wget https://package.cosmian.com/kms/4.21.2/ubuntu-24.04/cosmian-kms-server_4.21.0-1_amd64.deb
sudo apt install ./cosmian-kms-server_4.21.0-1_amd64.deb
cosmian_kms_server --version
```
Or install the FIPS version:
```sh
wget https://package.cosmian.com/kms/4.21.2/ubuntu-24.04/cosmian-kms-server-fips_4.21.0-1_amd64.deb
sudo apt install ./cosmian-kms-server-fips_4.21.0-1_amd64.deb
cosmian_kms_server --version
```
A `cosmian_kms` service will be configured; the service file is located at `/etc/systemd/system/cosmian_kms.service`.
The server will use a configuration file located at `/etc/cosmian_kms/kms.toml`.
To start the KMS, run:
```sh
sudo systemctl start cosmian_kms
```
=== "RHEL 9"
Download package and install it:
```sh
sudo dnf update && dnf install -y wget
wget https://package.cosmian.com/kms/4.21.2/rhel9/cosmian_kms_server-4.21.2-1.x86_64.rpm
sudo dnf install ./cosmian_kms_server-4.21.2-1.x86_64.rpm
cosmian_kms_server --version
```
=== "MacOS"
On ARM MacOS, download the build archive and extract it:
```sh
wget https://package.cosmian.com/kms/4.21.2/macos_arm-release.zip
unzip macos_arm-release.zip
cp ./macos_arm-release/Users/runner/work/kms/kms/target/aarch64-apple-darwin/release/cosmian_kms_server /usr/local/bin/
chmod u+x /usr/local/bin/cosmian_kms_server
cosmian_kms_server --version
```
On Intel MacOS, download the build archive and extract it:
```sh
wget https://package.cosmian.com/kms/4.21.2/macos_intel-release.zip
unzip macos_intel-release.zip
cp ./macos_intel-release/Users/runner/work/kms/kms/target/x86_64-apple-darwin/release/cosmian_kms_server /usr/local/bin/
chmod u+x /usr/local/bin/cosmian_kms_server
cosmian_kms_server --version
```
=== "Windows"
On Windows, download the build archive:
```sh
https://package.cosmian.com/kms/4.21.2/windows-release.zip
```
Extract the cosmian_kms_server from:
```sh
/windows-release/target/x86_64-pc-windows-msvc/release/cosmian_kms_server.exe
```
Copy it to a folder in your PATH and run it:
```sh
cosmian_kms_server --version
```

37
documentation/docs/marketplace_guide.md → documentation/docs/installation/marketplace_guide.md
View file

@ -1,12 +1,8 @@
# Deploy KMS in a Confidential Virtual Machine (CVM)
A KMS-ready instance based on Cosmian Confidential VM can be deployed on virtual machines
that supports AMD SEV-SNP or Intel TDX technologies, and is available on the marketplace of the major cloud providers.
A KMS-ready instance based on Cosmian VM can be deployed on virtual machines
that supports AMD SEV-SNP or Intel TDX technologies.
This instance can be deployed on virtual machines that supports AMD SEV-SNP or
Intel TDX technologies.
Please first read the guide about [how to setup a Cosmian VM](../cosmian_vm/deployment_guide.md).
If you are interested in the confidential computing technology and the Cosmian VM,
please first read the guide about [how to setup a Cosmian VM](../../cosmian_vm/deployment_guide.md).
## Deploy Cosmian VM KMS on a cloud provider
@ -34,7 +30,7 @@ The Cosmian KMS contains:
- a ready-to-go Nginx setup (listening on port `443` and locally on port `9998`)
- a ready-to-go KMS service
- the Cosmian VM software stack. As reminder, Cosmian VM Agent is listening
on port `5555`.
on port `5555`.
## Configure the KMS 📜
@ -44,9 +40,9 @@ By default:
- the KMS server is locally listening on port 9998
- its database is a local Redis database with encrypted data
using the scheme [Findex](../search/findex.md).
using the scheme [Findex](../../search/findex.md).
- the KMS configuration file is located in the encrypted LUKS container
at `/var/lib/cosmian_vm/data/app.conf` and has the following content:
at `/var/lib/cosmian_vm/data/app.conf` and has the following content:
```toml
default_username = "admin"
@ -62,7 +58,8 @@ redis_master_password = "master-password"
redis_findex_label = "label"
```
For testing purposes (connectivity, features, etc.), KMS server can also use a SQLite database by modifying the configuration file:
For testing purposes (connectivity, features, etc.), KMS server can also use a SQLite database by modifying the
configuration file:
```toml
default_username = "admin"
@ -84,7 +81,7 @@ hostname = "0.0.0.0"
### Override the default configuration
The default configuration can be overridden remotely by using the
[Cosmian VM CLI](../cosmian_vm/deployment_guide.md#install-the-cosmian-vm-cli)
[Cosmian VM CLI](../../cosmian_vm/deployment_guide.md#install-the-cosmian-vm-cli)
without any SSH connection.
It is safe to provide secrets (such as passwords) in
@ -92,11 +89,11 @@ the configuration file because this file is going to be stored in the encrypted
folder (LUKS) of the Cosmian VM KMS (which is mounted by default on `/var/lib/cosmian_vm/data`).
Cosmian VM CLI has to be installed on the client machine (Ubuntu, RHEL or via Docker).
Please follow the [installation instructions](../cosmian_vm/deployment_guide.md#install-the-cosmian-vm-cli).
Please follow the [installation instructions](../../cosmian_vm/deployment_guide.md#install-the-cosmian-vm-cli).
Then proceed as follows:
```console title="On the local machine"
```shell title="On the local machine"
cosmian_vm --url https://${COSMIAN_KMS_IP_ADDR}:5555 \
--allow-insecure-tls \
app init -c kms.toml
@ -111,7 +108,7 @@ contained in an encrypted container (LUKS).
where `kms.toml` can be:
```toml title="kms.toml"
```toml
default_username = "admin"
[http]
@ -126,11 +123,11 @@ redis_findex_label = "label"
```
- The database type `redis-findex` is a Redis database with encrypted data and
encrypted indexes thanks to Cosmian Findex.
encrypted indexes thanks to Cosmian Findex.
- The `database_url` points to the Redis, typically an external managed Redis database.
- The `redis_master_password` is used to encrypt the Redis data and indexes.
- The `redis_findex_label` is a public arbitrary label that can be changed
to rotate the Findex ciphertexts without changing the key.
to rotate the Findex ciphertexts without changing the key.
### Service
@ -177,7 +174,7 @@ information, creates self-signed certificate for Nginx and starts a snapshot.
Wait for the agent to initialize the LUKS and generate the certificates.
This is automatically at boot.
In short, to generate a snapshot, please [follow](../cosmian_vm/deployment_guide.md#snapshot-the-vm-remotely).
In short, to generate a snapshot, please [follow](../../cosmian_vm/deployment_guide.md#snapshot-the-vm-remotely).
The associated command is:
@ -188,7 +185,7 @@ cosmian_vm --url https://${COSMIAN_VM_IP_ADDR}:5555 --allow-insecure-tls snapsho
## Verify the Cosmian VM KMS integrity ✅
Verifying trustworthiness of the Cosmian VM KMS is exactly the same process
as [verifying the Cosmian VM](../cosmian_vm/overview.md) itself.
as [verifying the Cosmian VM](../../cosmian_vm/overview.md) itself.
In short, to verify a snapshot, please [follow](../cosmian_vm/deployment_guide.md#verify-the-vm-snapshot).

2
documentation/docs/kmip_2_1/attributes.md
View file

@ -45,7 +45,7 @@ All keys managed by the Cosmian KMS server are primarily a `KeyMaterial` made of
Typically a vendor attribute is made of 3 values: a `Vendor Identification` - always hardcoded to `cosmian` - and a tuple `Attribute Name`, `Attribute Value`.
Covercrypt uses a few vendor attributes which names can be seen in the code [attributes.rs](https://github.com/Cosmian/kms/blob/main/crate/kmip/src/crypto/cover_crypt/attributes.rs) file.
Covercrypt uses a few vendor attributes which names can be seen in the code [attributes.rs](https://github.com/Cosmian/kms/blob/main/crate/crypto/src/crypto/cover_crypt/attributes.rs) file.
The attributes names and corresponding values used for a given `KeyFormatType` are as follows:

24
documentation/docs/ms_dke/index.md
View file

@ -1,6 +1,7 @@
# Microsoft Double Key Encryption (DKE)
Microsoft Double Key Encryption (DKE) is a [feature of Microsoft 365](https://learn.microsoft.com/en-us/purview/double-key-encryption)
Microsoft Double Key Encryption (DKE) is
a [feature of Microsoft 365](https://learn.microsoft.com/en-us/purview/double-key-encryption)
that allows you to protect your most sensitive
data by encrypting data on the client computer before sending it to Microsoft servers.
One of the keys used to encrypt remains under your control and makes the data unreadable by Microsoft. This key is kept
@ -59,14 +60,15 @@ Alternatively, you can set the `KMS_MS_DKE_SERVICE_URL` environment variable to
corresponding entry in the server TOML configuration file.
!!! warning "No authentication => firewalling is critical"
The Office client does not send any authentication information when calling the Cosmian KMS. Firewalling the
Cosmian KMS server to only accept requests from valid Office clients is critical.
The Office client does not send any authentication information when calling the Cosmian KMS. Firewalling the
Cosmian KMS server to only accept requests from valid Office clients is critical.
!!! important "Running the KMS server in the cloud for DKE"
It is possible to confidentially run the Cosmian KMS server in the cloud [inside a
Cosmian VM](../marketplace_guide.md). However, due to the lack of authentication, and thus the need to firewall the server,
one should make sure to use OS-level firewalling and not rely on the cloud provider's firewalling capabilities,
particularly if running on Azure.
It is possible to confidentially run the Cosmian KMS server in the cloud [inside a
Cosmian VM](../installation/marketplace_guide.md). However, due to the lack of authentication, and thus the need to
firewall the server,
one should make sure to use OS-level firewalling and not rely on the cloud provider's firewalling capabilities,
particularly if running on Azure.
### Create an RSA key with tag `dke_key`
@ -77,11 +79,13 @@ cosmian kms rsa keys create --tag dke_key --size_in_bits 2048
```
The tag can be changed to any value, but it must be used in the URL of the sensitivity label in the Microsoft Purview
compliance portal. See [Create a sensitivity label for encryption](#create-a-sensitivity-label-for-encryption) for details.
compliance portal. See [Create a sensitivity label for encryption](#create-a-sensitivity-label-for-encryption) for
details.
#### Rotate the DKE key
If later on you need to rotate the DKE key, you can use the [Cosmian CLI](../../cosmian_cli/index.md) to create a new key with a new tag.
If later on you need to rotate the DKE key, you can use the [Cosmian CLI](../../cosmian_cli/index.md) to create a new
key with a new tag.
You must then create a new sensitivity label where the Double Key Encryption URL ends with the new tag value.
See [Create a sensitivity label for encryption](#create-a-sensitivity-label-for-encryption) for details.
@ -190,7 +194,7 @@ Select `Use Double Key Encryption` on the encryption configuration screen and ma
you do not activate co-authoring.
!!! important "Use the correct URL"
The URL must be set to a form similar to `https://dke.acme.com/ms_dke/dke_key` where
The URL must be set to a form similar to `https://dke.acme.com/ms_dke/dke_key` where
- `dke.acme.com` is the address of the Cosmian KMS server. A valid certificate must be installed on the server.
- `ms_dke` is the root of REST path for the DKE services.

39
documentation/docs/quick_start.md Normal file
View file

@ -0,0 +1,39 @@
# Quick start
To quick-start a Cosmian KMS server on `http://localhost:9998` that stores its data
inside the container, simply run the following command:
```sh
docker run -p 9998:9998 --name kms ghcr.io/cosmian/kms:latest
```
If you do not have Docker available, install a binary for your platform
from [Cosmian packages](https://package.cosmian.com/kms/).
Get the [Cosmian CLI](../cosmian_cli/index.md) from [Cosmian packages](https://package.cosmian.com/cli/).
You can then easily manage the server:
1) Create a 256-bit symmetric key
```sh
cosmian kms sym keys create --number-of-bits 256 --algorithm aes --tag my-file-key
...
The symmetric key was successfully generated.
Unique identifier: 87e9e2a8-4538-4701-aa8c-e3af94e44a9e
```
2) Encrypt the `image.png` file with AES GCM using the key
```sh
cosmian kms sym encrypt --tag my-file-key --output-file image.enc image.png
...
The encrypted file is available at "image.enc"
```
3) Decrypt the `image.enc` file using the key
```sh
cosmian kms sym decrypt --tag my-file-key --output-file image2.png image.enc
...
The decrypted file is available at "image2.png"
```

13
documentation/docs/server_cli.md
View file

@ -1,16 +1,11 @@
# Comprehensive inline help
When no [configuration file](./server_configuration_file.md) is provided, the KMS server can be
configured using command line options.
Just like the [Cosmian CLI](../cosmian_cli/index.md), the KMS server has a built-in help
system that can be accessed using the `--help` command line option.
The list of arguments can be printed using the `--help` command line option.
```sh
docker run --rm ghcr.io/cosmian/kms:latest --help
```
-> docker run --rm ghcr.io/cosmian/kms:latest --help
The options are enabled on the docker command line or using the environment variables listed in the
options help.
```text
Cosmian Key Management Service
Usage: cosmian_kms_server [OPTIONS]

114
documentation/docs/server_configuration_file.md
View file

@ -1,49 +1,109 @@
# TOML configuration file
The KMS server can be configured using a TOML file. When a configuration file is provided,
the [command line arguments](./server_cli.md) are ignored.
By default, the configuration filepath is retrieved in the following order:
1. if the environment variable `COSMIAN_KMS_CONF` is set and the path behind exists, the KMS server will use it as configuration file path.
2. otherwise if a file is found at `/etc/cosmian_kms/kms.toml`, the KMS server will use it to configure itself.
3. finally, if none of the above is found, the KMS server will load default configuration values in combination additional CLI arguments.
1. if the environment variable `COSMIAN_KMS_CONF` is set and the path behind exists, the KMS server will use this
configuration file,
2. otherwise if a file is found at `/etc/cosmian_kms/kms.toml`, the KMS server will use this file.
3. finally, if none of the above is found, the KMS server will use the [command line arguments](./server_cli.md)
The file should be a TOML file with the following structure:
```toml
default_username = "[default username]"
# The default username to use when no authentication method is provided
default_username = "admin"
# When an authentication method is provided, perform the authentication
# but always use the default username instead of the one provided by the authentication method
force_default_username = false
google_cse_kacls_url = "[google cse kacls url]"
ms_dke_service_url = "[ms dke service url]"
info = false
hsm_model = "proteccio"
hsm_admin = "[hsm admin username]" #for Create operation on HSM
hsm_slot = [number_of_slot1, number_of_slot2, ...]
hsm_password = [password_of_slot1, password_of_slot2, ...]
# This setting enables the Google Workspace Client Side Encryption feature of this KMS server.
# It should contain the external URL of this server as configured
# in Google Workspace client side encryption settings For instance,
# if this server is running on domain `cse.my_domain.com`,
# the URL should be something like <https://cse.my_domain.com/google_cse>
google_cse_kacls_url = "<google cse kacls url>"
# This setting disables the validation of the tokens used by the Google Workspace CSE feature of this server
# Usefeull for testing purposes
google-cse-disable-tokens-validation = false
# This setting enables the Microsoft Double Key Encryption service feature of this server.
# It should contain the external URL of this server as configured in Azure App Registrations
# as the DKE Service (<https://learn.microsoft.com/en-us/purview/double-key-encryption-setup#register-your-key-store>)
# The URL should be something like <https://cse.my_domain.com/ms_dke>
ms_dke_service_url = "<ms dke service url>"
# Print the server configuration information and exit
info = false
# The following fields are only needed if an HSM is used.
# Check the HSMs pages for more information.
hsm_model = "<hsm_name>"
hsm_admin = "<hsm admin username>" #for Create operation on HSM
hsm_slot = [1, 2, ...]
hsm_password = ["<password_of_1st_slot1>", "<password_of_2bd_slot2>", ...]
# Check the database pages for more information
[db]
database_type = "[redis-findex, postgresql,...]"
database_url = "[redis urls]"
sqlite_path = "[sqlite path]"
redis_master_password = "[redis master password]"
redis_findex_label = "[redis findex label]"
database_type = "postgresql", "mysql", "sqlite", "sqlite-enc", "redis-findex"
database_url = "<database-url>"
sqlite_path = "<sqlite-path>"
redis_master_password = "<redis master password>"
redis_findex_label = "<redis findex label>"
clear_database = false
# Check the Enabling TLS pages for more information
[http]
port = 443
hostname = "[hostname]"
https_p12_file = "[https p12 file]"
https_p12_password = "[https p12 password]"
authority_cert_file = "[authority cert file]"
# The KMS server port - defaults to 9998
port = 9998
# The KMS server hostname - defaults to 0.0.0.0
hostname = "<hostname>"
# The KMS server optional PKCS#12 Certificates and Key file.
# If provided, this will start the server in HTTPS mode
https_p12_file = "<https p12 file>"
# The password to open the PKCS#12 Certificates and Key file
https_p12_password = "<https p12 password>"
# The server optional authority X509 certificate in PEM format
# used to validate the client certificate presented for authentication.
# If provided, this will require clients to present a certificate signed by this authority for authentication.
# The server must run in TLS mode for this to be used
authority_cert_file = "<authority cert file>"
# Check the Auhtenticating Users for more information
[auth]
jwt_issuer_uri = ["[jwt issuer uri]"]
jwks_uri = ["[jwks uri]"]
jwt_audience = ["[jwt audience]"]
# The issuer URI of the JWT token
# To handle multiple identity managers, add different parameters
# under each argument (jwt-issuer-uri, jwks-uri and optionally jwt-audience),
# keeping them in the same order in the three lists.
# For Auth0, this is the delegated authority domain configured on Auth0, for instance `https://<your-tenant>.<region>.auth0.com/`
# For Google, this would be `https://accounts.google.com`
jwt_issuer_uri = ["<jwt issuer uri>"]
# The JWKS (Json Web Key Set) URI of the JWT token
# To handle multiple identity managers, add different parameters under each argument
# (jwt-issuer-uri, jwks-uri and optionally jwt-audience), keeping them in the same order
# For Auth0, this would be `https://<your-tenant>.<region>.auth0.com/.well-known/jwks.json`
# For Google, this would be `https://www.googleapis.com/oauth2/v3/certs`
# Defaults to `<jwt-issuer-uri>/.well-known/jwks.json` if not set
jwks_uri = ["<jwks uri>"]
# The audience of the JWT token
# Optional: the server will validate the JWT `aud` claim against this value if set
jwt_audience = ["<jwt audience>"]
[workspace]
root_data_path = "[root data path]"
tmp_path = "[tmp path]"
# The root folder where the KMS will store its data
# A relative path is taken relative to the user HOME directory
root_data_path = "./cosmian-kms"
# The folder to store temporary data (non-persistent data readable
# by no-one but the current instance during the current execution)
tmp_path = "/tmp"
# Check the logging pages for more information
[telemetry]
otlp = "[url of the OTLP collector]"
# The Open Telemetry OTLP collector URL
otlp = "<url of the OTLP collector>"
# Do not log to stdout
quiet = false
```

216
documentation/docs/single_server_mode.md
View file

@ -1,216 +0,0 @@
The single server mode uses an embedded SQLite database stored on a filesystem and therefore does
not require access to an external database.
Although it does not provide high availability through redundancy, this configuration is suitable
for production and serving millions of cryptographic objects. The server will concurrently serve
requests on as many threads as available cores to the docker container.
This configuration also supports user encrypted databases, a secure way to store cryptographic
objects since database keys are provisioned on every request, and no database key is stored server
side. To offer a fully secure solution suitable for deployment in a zero-trust environment such as
the cloud, TLS must be enabled on the server, and the memory of the KMS server must also be
protected during operation by running the server inside an enclave. Ask Cosmian for details.
### Quick start
To run in single server mode, using the defaults and a SQLite database will be created. Otherwise,
the database can be configured using classic databases such as PostgreSQL, MySQL or MariaDB or the Cosmian custom protected Redis, please follow [the database configuration page](./database.md).
=== "Docker"
Run the container as follows:
```sh
docker run -p 9998:9998 --name kms ghcr.io/cosmian/kms:latest
```
The KMS will be available on `http://localhost:9998`, and the server will store its data inside the
container in the `/root/cosmian-kms/sqlite-data` directory.
FIPS version is also available:
```sh
docker run -p 9998:9998 --name kms ghcr.io/cosmian/kms-fips:latest
```
To persist data between restarts, map the `/root/cosmian-kms/sqlite-data` path to a filesystem
directory or a Docker volume, e.g. with a volume named `cosmian-kms`:
```sh
docker run --rm -p 9998:9998 \
-v cosmian-kms:/root/cosmian-kms/sqlite-data \
--name kms ghcr.io/cosmian/kms:latest
```
=== "Ubuntu 20.04"
Download package and install it:
```console title="On local machine"
sudo apt update && sudo apt install -y wget
wget https://package.cosmian.com/kms/4.21.2/ubuntu-20.04/cosmian-kms-server_4.21.0-1_amd64.deb
sudo apt install ./cosmian-kms-server_4.21.0-1_amd64.deb
cosmian_kms_server --version
```
Or install the FIPS version:
```console title="FIPS version"
wget https://package.cosmian.com/kms/4.21.2/ubuntu-20.04/cosmian-kms-server-fips_4.21.0-1_amd64.deb
sudo apt install ./cosmian-kms-server-fips_4.21.0-1_amd64.deb
cosmian_kms_server --version
```
=== "Ubuntu 22.04"
Download package and install it:
```console title="On local machine"
sudo apt update && sudo apt install -y wget
wget https://package.cosmian.com/kms/4.21.2/ubuntu-22.04/cosmian-kms-server_4.21.0-1_amd64.deb
sudo apt install ./cosmian-kms-server_4.21.0-1_amd64.deb
cosmian_kms_server --version
```
Or install the FIPS version:
```console title="FIPS version"
wget https://package.cosmian.com/kms/4.21.2/ubuntu-22.04/cosmian-kms-server-fips_4.21.0-1_amd64.deb
sudo apt install ./cosmian-kms-server-fips_4.21.0-1_amd64.deb
cosmian_kms_server --version
```
=== "Ubuntu 24.04"
Download package and install it:
```console title="On local machine"
sudo apt update && sudo apt install -y wget
wget https://package.cosmian.com/kms/4.21.2/ubuntu-24.04/cosmian-kms-server_4.21.0-1_amd64.deb
sudo apt install ./cosmian-kms-server_4.21.0-1_amd64.deb
cosmian_kms_server --version
```
Or install the FIPS version:
```console title="FIPS version"
wget https://package.cosmian.com/kms/4.21.2/ubuntu-24.04/cosmian-kms-server-fips_4.21.0-1_amd64.deb
sudo apt install ./cosmian-kms-server-fips_4.21.0-1_amd64.deb
cosmian_kms_server --version
```
=== "RHEL 9"
Download package and install it:
```console title="On local machine"
sudo dnf update && dnf install -y wget
wget https://package.cosmian.com/kms/4.21.2/rhel9/cosmian_kms_server-4.21.2-1.x86_64.rpm
sudo dnf install ./cosmian_kms_server-4.21.2-1.x86_64.rpm
cosmian_kms_server --version
```
=== "MacOS"
On ARM MacOS, download the build archive and extract it:
```console title="On local machine"
wget https://package.cosmian.com/kms/4.21.2/macos_arm-release.zip
unzip macos_arm-release.zip
cp ./macos_arm-release/Users/runner/work/kms/kms/target/aarch64-apple-darwin/release/cosmian_kms_server /usr/local/bin/
chmod u+x /usr/local/bin/cosmian_kms_server
cosmian_kms_server --version
```
On Intel MacOS, download the build archive and extract it:
```console title="On local machine"
wget https://package.cosmian.com/kms/4.21.2/macos_intel-release.zip
unzip macos_intel-release.zip
cp ./macos_intel-release/Users/runner/work/kms/kms/target/x86_64-apple-darwin/release/cosmian_kms_server /usr/local/bin/
chmod u+x /usr/local/bin/cosmian_kms_server
cosmian_kms_server --version
```
=== "Windows"
On Windows, download the build archive:
```console title="Build archive"
https://package.cosmian.com/kms/4.21.2/windows-release.zip
```
Extract the cosmian_kms_server from:
```console title="cosmian_kms_server for Windows"
/windows-release/target/x86_64-pc-windows-msvc/release/cosmian_kms_server.exe
```
Copy it to a folder in your PATH and run it:
```console title="On local machine"
cosmian_kms_server --version
```
### Using client-side encrypted databases
To start the KMS server with a client-side encrypted SQLite databases, pass the
`--database-type=sqlite-enc` on start, e.g.
```sh
docker run --rm -p 9998:9998 \
-v cosmian-kms:/root/cosmian-kms/sqlite-data \
--name kms ghcr.io/cosmian/kms:latest \
--database-type=sqlite-enc
```
It requires now to install the [Cosmian CLI](../cosmian_cli/index.md) and create a new encrypted database.
!!! important "Important: encrypted databases must be created first"
Before using an encrypted database, you must create it by calling the `POST /new_database` endpoint.
The call will return a secret
=== "cosmian"
```sh
cosmian kms new-database
```
=== "curl"
```sh
➜ curl -X POST https://my-server:9998/new_database
"eyJncm91cF9pZCI6MzE0ODQ3NTQzOTU4OTM2Mjk5OTY2ODU4MTY1NzE0MTk0MjU5NjUyLCJrZXkiOiIzZDAyNzg3YjUyZGY5OTYzNGNkOTVmM2QxODEyNDk4YTRiZWU1Nzc1NmM5NDI0NjdhZDI5ZTYxZjFmMmM0OWViIn0="%
```
The secret is the value between the quotes `""`.
Warning:
- This secret is only displayed **once** and is **not stored** anywhere on the server.
- Each call to `new_database` will create a **new additional** database. It will not return the secret of the last created database, and it will not overwrite the last created database.
Once an encrypted database is created, the secret must be passed in every subsequent query to the
KMS server.
Passing the correct secret "auto-selects" the correct encrypted database: multiple encrypted
databases can be used concurrently on the same KMS server.
=== "cosmian"
The secret must be set in `database_secret` property of the CLI `cosmian.json` configuration file.
```toml
[kms_config.http_config]
server_url = "http://127.0.0.1:9990"
access_token = "eyJhbGciOiJSUzI1NiIsInR5cCI6Ik...yaJbDDql3A"
database_secret = "eyJncm91cF9pZCI6MTI5N...MWIwYjE5ZmNlN2U3In0="
```
=== "curl"
The secret must be passed using a `DatabaseSecret` HTTP header, e.g.
```sh
curl \
-H "DatabaseSecret: eyJncm91cF9pZCI6MzE0ODQ3NTQzOTU4OTM2Mjk5OTY2ODU4MTY1NzE0MTk0MjU5NjUyLCJrZXkiOiIzZDAyNzg3YjUyZGY5OTYzNGNkOTVmM2QxODEyNDk4YTRiZWU1Nzc1NmM5NDI0NjdhZDI5ZTYxZjFmMmM0OWViIn0=" \
http://localhost:9998/objects/owned
```

27
documentation/docs/tls.md
View file

@ -1,7 +1,7 @@
The server can serve requests using either plaintext HTTP or HTTPS.
When running in a zero-trust environment, the KMS server should be started using HTTPS.
Check the [running in a zero-trust environment](./marketplace_guide.md) section for more information.
Check the [running in a zero-trust environment](installation/marketplace_guide.md) section for more information.
To enable TLS, one can provide certificates on the command line interface.
@ -18,17 +18,22 @@ There are 2 ways to provide the PKCS#12 file to the server:
Specify the certificate name and mount the file to docker.
Say the certificate is called `server.mydomain.com.p12`, is protected by the password `myPass`, and is in a directory called `/certificate` on the host disk.
Say the certificate is called `server.mydomain.com.p12`, is protected by the password `myPass`, and is in a directory
called `/certificate` on the host disk.
```sh
docker run --rm -p 443:9998 \
-v /certificate/server.mydomain.com.p12:/root/cosmian-kms/server.mydomain.com.p12 \
--name kms ghcr.io/cosmian/kms:latest \
--database-type=mysql \
--database-url=mysql://mysql_server:3306/kms \
--https-p12-file=server.mydomain.com.p12 \
--https-p12-password=myPass
```
=== "kms.toml"
```toml
[http]
https-p12-file="<server.mydomain.com.p12>"
https-p12-password="<myPass>"
```
=== "Command line arguments"
```sh
--https-p12-file=server.mydomain.com.p12 \
--https-p12-password=myPass
```
!!!info "Generate a PKCS#12 from PEM files"
To generate a PKCS12 from PEM files, you can use `openssl`:

26
documentation/mkdocs.yml
View file

@ -58,8 +58,9 @@ plugins:
- kroki
- meta-descriptions
nav:
- Getting started: index.md
- Use cases:
- Why use the Cosmian KMS: index.md
- Quick start: quick_start.md
- Use cases and integrations:
- Encrypting and decrypting at scale: encrypting_and_decrypting_at_scale.md
- Google workspace Client-Side Encryption (CSE):
- Getting started: google_cse/index.md
@ -68,25 +69,28 @@ nav:
- Migrating existing email to Gmail CSE: google_cse/migrating.md
- Microsoft Double Key Encryption (DKE): ms_dke/index.md
- HSM support:
- Proteccio: hsm.md
- Introduction: hsms/index.md
- HSM keys & operations: hsms/hsm_operations.md
- Trustway Proteccio: hsms/proteccio.md
- Utimaco General Purpose: hsms/utimaco.md
- Disk Encryption:
- Veracrypt: pkcs11/veracrypt.md
- LUKS: pkcs11/luks.md
- Cryhod: pkcs11/cryhod.md
- S/MIME Email encryption: pki/smime.md
- API Endpoints: api.md
- Server Installation:
- Single server mode: single_server_mode.md
- High-availability: high_availability_mode.md
- Deploying in Confidential VM: marketplace_guide.md
- Server Configuration:
- Command Line Interface: server_cli.md
- Installation:
- Getting started: installation/installation_getting_started.md
- Deploying in a Cosmian Confidential VM: installation/marketplace_guide.md
- High-availability: installation/high_availability_mode.md
- Configuration:
- Configuration file: server_configuration_file.md
- Command line arguments: server_cli.md
- Databases: database.md
- Authenticating users to the server: authentication.md
- Authorizing users with access rights: authorization.md
- Enabling TLS: tls.md
- Logging and telemetry: logging.md
- Database: database.md
- S/MIME Email encryption: pki/smime.md
- Certifications and compliance:
- FIPS 140-3: fips.md
- Cryptographic algorithms: