AWS CodeArtifact adds support for Rust packages with Cargo
Starting today, Rust developers can store and access their libraries (known as crates in Rust’s world) on AWS CodeArtifact.
Modern software development relies heavily on pre-written code packages to accelerate development. These packages, which can number in the hundreds for a single application, tackle common programming tasks and can be created internally or obtained from external sources. While these packages significantly help to speed up development, their use introduces two main challenges for organizations: legal and security concerns.
On the legal side, organizations need to ensure they have compatible licenses for these third-party packages and that they don’t infringe on intellectual property rights. Security is another risk, as vulnerabilities in these packages could be exploited to compromise an application. A known tactic, the supply chain attack, involves injecting vulnerabilities into popular open source projects.
To address these challenges, organizations can set up private package repositories. These repositories store pre-approved packages vetted by security and legal teams, limiting the risk of legal or security exposure. This is where CodeArtifact enters.
AWS CodeArtifact is a fully managed artifact repository service designed to securely store, publish, and share software packages used in application development. It supports popular package managers and formats such as npm, PyPI, Maven, NuGet, SwiftPM, and Rubygem, enabling easy integration into existing development workflows. It helps enhance security through controlled access and facilitates collaboration across teams. CodeArtifact helps maintain a consistent, secure, and efficient software development lifecycle by integrating with AWS Identity and Access Management (IAM) and continuous integration and continuous deployment (CI/CD) tools.
For the eighth year in a row, Rust has topped the chart as “the most desired programming language” in Stack Overflow’s annual developer survey, with more than 80 percent of developers reporting that they’d like to use the language again next year. Rust’s growing popularity stems from its ability to combine the performance and memory safety of systems languages such as C++ with features that makes writing reliable, concurrent code easier. This, along with a rich ecosystem and a strong focus on community collaboration, makes Rust an attractive option for developers working on high-performance systems and applications.
Rust developers rely on Cargo, the official package manager, to manage package dependencies. Cargo simplifies the process of finding, downloading, and integrating pre-written crates (libraries) into their projects. This not only saves time by eliminating manual dependency management, but also ensures compatibility and security. Cargo’s robust dependency resolution system tackles potential conflicts between different crate versions, and because many crates come from a curated registry, developers can be more confident about the code’s quality and safety. This focus on efficiency and reliability makes Cargo an essential tool for building Rust applications.
Let’s create a CodeArtifact repository for my crates
In this demo, I use the AWS Command Line Interface (AWS CLI) and AWS Management Console to create two repositories. I configure the first repository to download public packages from the official crates.io repository. I configure the second repository to download packages from the first one only. This dual repository configuration is the recommended way to manage repositories and external connections, see the CodeArtifact documentation for managing external connections. To quote the documentation:
“It is recommended to have one repository per domain with an external connection to a given public repository. To connect other repositories to the public repository, add the repository with the external connection as an upstream to them.”
I sketched this diagram to illustrate the setup.
Domains and repositories can be created either from the command line or the console. I choose the command line. In shell terminal, I type:
CODEARTIFACT_DOMAIN=stormacq-test
# Create an internal-facing repository: crates-io-store
aws codeartifact create-repository
--domain $CODEARTIFACT_DOMAIN
--repository crates-io-store
# Associate the internal-facing repository crates-io-store to the public crates-io
aws codeartifact associate-external-connection
--domain $CODEARTIFACT_DOMAIN
--repository crates-io-store
--external-connection public:crates-io
# Create a second internal-facing repository: cargo-repo
# and connect it to upstream crates-io-store just created
aws codeartifact create-repository
--domain $CODEARTIFACT_DOMAIN
--repository cargo-repo
--upstreams '{"repositoryName":"crates-io-store"}' Next, as a developer, I want my local machine to fetch crates from the internal repository (cargo-repo) I just created.
I configure cargo to fetch libraries from the internal repository instead of the public crates.io. To do so, I create a config.toml file to point to CodeArtifact internal repository.
# First, I retrieve the URI of the repo
REPO_ENDPOINT=$(aws codeartifact get-repository-endpoint
--domain $CODEARTIFACT_DOMAIN
--repository cargo-repo
--format cargo
--output text)
# at this stage, REPO_ENDPOINT is https://stormacq-test-012345678912.d.codeartifact.us-west-2.amazonaws.com/cargo/cargo-repo/
# Next, I create the cargo config file
cat << EOF > ~/.cargo/config.toml
[registries.cargo-repo]
index = "sparse+$REPO_ENDPOINT"
credential-provider = "cargo:token-from-stdout aws codeartifact get-authorization-token --domain $CODEARTIFACT_DOMAIN --query authorizationToken --output text"
[registry]
default = "cargo-repo"
[source.crates-io]
replace-with = "cargo-repo"
EOF
Note that the two environment variables are replaced when I create the config file. cargo doesn’t support environment variables in its configuration.
From now on, on this machine, every time I invoke cargo to add a crate, cargo will obtain an authorization token from CodeArtifact to communicate with the internal cargo-repo repository. I must have IAM privileges to call the get-authorization-token CodeArtifact API in addition to permissions for read/publish package according to the command I use. If you’re running this setup from a build machine for your continuous integration (CI) pipeline, your build machine must have proper permissions to do so.
I can now test this setup and add a crate to my local project.
$ cargo add regex
Updating `codeartifact` index
Adding regex v1.10.4 to dependencies
Features:
+ perf
+ perf-backtrack
+ perf-cache
+ perf-dfa
+ perf-inline
+ perf-literal
+ perf-onepass
+ std
+ unicode
+ unicode-age
+ unicode-bool
+ unicode-case
+ unicode-gencat
+ unicode-perl
+ unicode-script
+ unicode-segment
- logging
- pattern
- perf-dfa-full
- unstable
- use_std
Updating `cargo-repo` index
# Build the project to trigger the download of the crate
$ cargo build
Downloaded memchr v2.7.2 (registry `cargo-repo`)
Downloaded regex-syntax v0.8.3 (registry `cargo-repo`)
Downloaded regex v1.10.4 (registry `cargo-repo`)
Downloaded aho-corasick v1.1.3 (registry `cargo-repo`)
Downloaded regex-automata v0.4.6 (registry `cargo-repo`)
Downloaded 5 crates (1.5 MB) in 1.99s
Compiling memchr v2.7.2 (registry `cargo-repo`)
Compiling regex-syntax v0.8.3 (registry `cargo-repo`)
Compiling aho-corasick v1.1.3 (registry `cargo-repo`)
Compiling regex-automata v0.4.6 (registry `cargo-repo`)
Compiling regex v1.10.4 (registry `cargo-repo`)
Compiling hello_world v0.1.0 (/home/ec2-user/hello_world)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 16.60sI can verify CodeArtifact downloaded the crate and its dependencies from the upstream public repository. I connect to the CodeArtifact console and check the list of packages available in either repository I created. At this stage, the package list should be identical in the two repositories.
Publish a private package to the repository
Now that I know the upstream link works as intended, let’s publish a private package to my cargo-repo repository to make it available to other teams in my organization.
To do so, I use the standard Rust tool cargo, just like usual. Before doing so, I add and commit the project files to the gitrepository.
$ git add . && git commit -m "initial commit"
5 files changed, 1855 insertions(+)
create mode 100644 .gitignore
create mode 100644 Cargo.lock
create mode 100644 Cargo.toml
create mode 100644 commands.sh
create mode 100644 src/main.rs
$ cargo publish
Updating `codeartifact` index
Packaging hello_world v0.1.0 (/home/ec2-user/hello_world)
Updating crates.io index
Updating `codeartifact` index
Verifying hello_world v0.1.0 (/home/ec2-user/hello_world)
Compiling libc v0.2.155
... (redacted for brevity) ....
Compiling hello_world v0.1.0 (/home/ec2-user/hello_world/target/package/hello_world-0.1.0)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 03s
Packaged 5 files, 44.1KiB (11.5KiB compressed)
Uploading hello_world v0.1.0 (/home/ec2-user/hello_world)
Uploaded hello_world v0.1.0 to registry `cargo-repo`
note: waiting for `hello_world v0.1.0` to be available at registry `cargo-repo`.
You may press ctrl-c to skip waiting; the crate should be available shortly.
Published hello_world v0.1.0 at registry `cargo-repo`Lastly, I use the console to verify the hello_world crate is now available in the cargo-repo.
Pricing and availability
You can now store your Rust libraries in the 13 AWS Regions where CodeArtifact is available. There is no additional cost for Rust packages. The three billing dimensions are the storage (measured in GB per month), the number of requests, and the data transfer out to the internet or to other AWS Regions. Data transfer to AWS services in the same Region is not charged, meaning you can run your continuous integration and delivery (CI/CD) jobs on Amazon Elastic Compute Cloud (Amazon EC2) or AWS CodeBuild, for example, without incurring a charge for the CodeArtifact data transfer. As usual, the pricing page has the details.
Now go build your Rust applications and upload your private crates to CodeArtifact!
— seb
Source: AWS News