Writing a Rust gem from scratch

Note: anytime you see 🙋‍♂️ RFC, that’s a “Request For Comments” about a topic I didn’t understand or take the time to look into. Please feel free to add what you know!

This is a followup to the last post. Instead of using the template rust_ruby_example gem, we’ll make one from scratch. Make sure to go back over the “Using a rubygems fork” section because we’ll be using it heavily during this post, as well!

Requirements

Requirements / dependencies / utilities I used and their versions on macOS Monterey v12.1 (21C52) as of 2022-02-02:

  • Bundler version 2.4.0.dev
  • gem version 3.4.0.dev
  • cargo 1.58.0 (f01b232bc 2022-01-19)
  • rustc 1.58.1 (db9d1b20b 2022-01-20)

Generating a new gem

Let’s see what it takes to write a Rust gem from scratch. Thankfully, Bundler has a generator for making new gems, and we can look at the rust_ruby_example gem for pointers on how to get the Rust parts working.

We’re still going to be doing some string manipulation, but this time we’ll just shuffle the characters. Full disclosure: I also don’t know enough of Ruby’s C API to do much more than that. That’s an adventure for another day!

Let’s start out by asking bundler to make a new gem. We’ll call it rust_shuffle, or ruffle for short.

$ bundle gem ruffle
# …omitted
Gem 'ruffle' was successfully created. For more information on making a RubyGem visit https://bundler.io/guides/creating_gem.html
$ cd ruffle
view raw 000.sh hosted with ❤ by GitHub

If this is the first time you’ve created a new gem with bundler, it may ask you a few configuration questions first. These questions are saved to your user’s profile at ~/.bundle/config and can be changed with the bundle config subcommand. For our purposes, the only one that matters is using rspec as the test framework.

Speaking of rspec, let’s $ bundle install in our new ruffle directory so we can fetch the rspec gem:

$ bundle install
You have one or more invalid gemspecs that need to be fixed.
The gemspec at /Users/brian/Code/github/briankung/ruffle/ruffle.gemspec is not valid. Please fix this
gemspec.
The validation error was 'metadata['homepage_uri'] has invalid link: "TODO: Put your gem's website or
public repo URL here."'
view raw 001.sh hosted with ❤ by GitHub

Well! Looks like we’ll be using some good old-fashioned EDD (Error Driven Development) to get this gem ship-shape.

The problem here is that the gemspec, a metadata file, needs to be filled out before the gem is valid. We’re going to do it the easy way and simply remove all the TODO‘s from the. Bundler also checks that the URLs parse correctly, so we’ll be replacing all the URLs with "https://example.com":

# in ruffle.gemspec
Gem::Specification.new do |spec|
# …omitted
spec.summary = "Write a short summary, because RubyGems requires one."
spec.description = "Write a longer description or delete this line."
spec.homepage = "https://example.com"
# …omitted
spec.metadata["allowed_push_host"] = spec.homepage
spec.metadata["homepage_uri"] = spec.homepage
spec.metadata["source_code_uri"] = spec.homepage
spec.metadata["changelog_uri"] = spec.homepage
# …omitted
end
view raw 002.rb hosted with ❤ by GitHub

And now we can fetch our dependencies:

$ bundle
Fetching gem metadata from https://rubygems.org/…
Resolving dependencies…
# …omitted
Fetching rspec 3.10.0
Installing rspec 3.10.0
Bundle complete! 3 Gemfile dependencies, 9 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
view raw 003.sh hosted with ❤ by GitHub

Now, what happens when we run our specs?

$ rake # defaults to rake spec
Ruffle
has a version number
does something useful (FAILED – 1)
Failures:
1) Ruffle does something useful
Failure/Error: expect(false).to eq(true)
expected: true
got: false
(compared using ==)
Diff:
@@ -1 +1 @@
-true
+false
# ./spec/ruffle_spec.rb:9:in `block (2 levels) in <top (required)>'
# /Users/brian/Code/github/ianks/rubygems/bundler/spec/support/bundle.rb:8:in `load'
# /Users/brian/Code/github/ianks/rubygems/bundler/spec/support/bundle.rb:8:in `<main>'
Finished in 0.00947 seconds (files took 0.07426 seconds to load)
2 examples, 1 failure
Failed examples:
rspec ./spec/ruffle_spec.rb:8 # Ruffle does something useful
view raw 004.sh hosted with ❤ by GitHub

The test “Ruffle does something useful” in the file spec/ruffle_spec.rb on line 8 fails. Harsh.

Adding a #shuffle method in Rust

First, let’s change the “does something useful” test to test something useful. Replace spec/ruffle_spec.rb with the following:

# in spec/ruffle_spec.rb
RSpec.describe Ruffle do
it "has a #shuffle method" do
expect(Ruffle).to respond_to(:shuffle)
end
end
view raw 005.rb hosted with ❤ by GitHub

Now let’s run the test, like good EDD practitioners – red, green, deploy. Then right back to EDD, this time in prod (it’s a virtuous cycle):

Failures:
1) Ruffle has a #shuffle method
Failure/Error: expect(Ruffle).to respond_to(:shuffle)
expected Ruffle to respond to :shuffle
view raw 006.sh hosted with ❤ by GitHub

You may see a warning like --pattern spec/**{,/*/**}/*_spec.rb failed. This is a result of running rspec through rake. While it’s annoying, it’s not a showstopper.

Initializing a Rust project

Now let’s add some Rust code! For the purposes of this tutorial, we’ll add it directly to the root directory of the gem, but there is a standard project structure for gem native extensions.

(🙋‍♂️ RFC: should a Rust extension follow the rake-compiler project structure?)

You can initialize a Rust project with:

$ cargo init –lib
Created library package
view raw 007.sh hosted with ❤ by GitHub

We pass the --lib argument to cargo to tell it that we want our crate to be a library, not a binary. Whereas a binary results in an executable, a library crate’s output should be something like .so or .dll files, if not compiled directly into another Rust binary.

Now we have two new files: Cargo.toml and src/lib.rs. cargo has also modified the .gitignore file to ignore build artifacts and Cargo.lock, a lockfile for Cargo dependencies.

Adding a rake compile task

Let’s add a rake task to compile our Rust code. Rakefiles are Ruby files that define the tasks that rake can run. If we inspect ours, it looks fairly barebones:

# in Rakefile
require "bundler/gem_tasks"
require "rspec/core/rake_task"
RSpec::Core::RakeTask.new(:spec)
task default: :spec
view raw 008.rb hosted with ❤ by GitHub

Basically it requires some default tasks. It also defines the :spec task as the default task, or the task that’s run when you execute rake without any arguments. We’ve already used it, in fact.

Let’s add a compile task. Make sure the RUBYGEMS_PATH environment variable is set from the last post. If you don’t have it, make sure to export it:

$ export RUBYGEMS_PATH=path/to/your/cargo-builder/rubygems
view raw 013.sh hosted with ❤ by GitHub

Now we can reference that:

# in Rakefile
desc "Compile the ruffle crate"
task :compile do
cargo_builder_gem = [
"ruby",
"-I#{ENV["RUBYGEMS_PATH"]}/lib",
"#{ENV["RUBYGEMS_PATH"]}/bin/gem"
]
gemspec = File.expand_path('ruffle.gemspec')
output = File.expand_path('ruffle.gem')
`gem list -i "^ruffle$"`
gem_installed = Process.last_status.success?
system *cargo_builder_gem, 'uninstall', 'ruffle' if gem_installed
system *cargo_builder_gem, 'build', gemspec, '–output', output
system *cargo_builder_gem, 'install', output
end
view raw 014-3.rb hosted with ❤ by GitHub
💡 Click for additional notes – *cargo_builder_gem?

We have to make sure we’re using the cargo-builder branch of rubygems. If we simply shell out to gem, we’ll end up using our default system gem.

I was hoping to use the rubygem internals to build the gem instead of relying on shelling out to the utility. I got as far as to require 'rubygems/ext' in order to use the Gem::Ext::CargoBuilder class, but realized that gem and bundler are just aliases to the cargo-builder branch of our rubygems repo and not a system-wide installation, I don’t have access to it within Ruby itself. It will be much easier once the PR is merged into the repository proper.

Now, there is a way using the setup.rb script in the rubygems repository, but it requires replacing your default rubygems gems. As long as this article is, it would be even longer with the caveats and restoration that would take, so I chose not to use setup.rb.


And now to test:

$ rake compile
true
Successfully uninstalled ruffle-0.1.0
Successfully built RubyGem
Name: ruffle
Version: 0.1.0
File: ruffle.gem
Successfully installed ruffle-0.1.0
1 gem installed
view raw 015.sh hosted with ❤ by GitHub
💡 Click for additional notes – "true"?

EDIT – Thanks Zach! I’ve changed it to backticks 😊

The call to check whether the ruffle gem is installed is system 'gem', 'list', '-i', '^ruffle$'. Unfortunately, the actual shell command prints true or false to stdout, so I can’t swallow it by, for example, assigning it to a variable. I’m too lazy and this post has taken too long already to add this to the list of things to figure out.

My apologies 🙇‍♂️

🙋‍♂️ RFC: what’s a better way to do this?


At this point, however, it’s not actually building the extension. If it were, it would print the message "Building native extensions. This could take a while...".

Fixing crate compilation

There are two more things we need to change in ruffle.gemspec before it’ll work. First, we have to add "Cargo.toml" to spec.extensions:

# in ruffle.gemspec
Gem::Specification.new do |spec|
# …omitted
spec.required_ruby_version = ">= 2.6.0"
# 👇 Add this
spec.extensions = ["Cargo.toml"]
# …
end
view raw 016.rb hosted with ❤ by GitHub

But this still isn’t enough. If we run $ rake compile now, we get a cryptic error message:

error: the lock file /Users/brian/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/ruffle-0.1.0/Cargo.lock needs to be updated but –locked was passed to prevent this
If you want to try to generate the lock file without accessing the network, remove the –locked flag and use –offline instead.
view raw 017.sh hosted with ❤ by GitHub

The problem is that the we aren’t telling the gemspec to package the Cargo.lock file in the final .gem file, and building the crate requires a lock on the Cargo.lock file. So how do we add the lockfile to the finished gem? By adding them to the spec.files array in the gemspec. You can read more about that here.

There are two ways we can do this:

  1. Add the Cargo.lock and Cargo.toml files to the git history. The default logic for spec.files relies on the git history. It uses git ls-files -z.split("\x0"), which only reports files that have been added to git.
  2. Specifically add ["Cargo.lock", "Cargo.toml", and "src/lib.rs"] to spec.files in the gemspec.

Since we haven’t been paying much attention to git and this is a simple enough project with no Ruby files, we’ll go with option #2. In ruffle.gemspec, find the code that assigns to spec.files:

# in ruffle.gemspec
spec.files = Dir.chdir(File.expand_path(__dir__)) do
`git ls-files -z`.split("\x0").reject do |f|
(f == __FILE__) || f.match(%r{\A(?:(?:bin|test|spec|features)/|\.(?:git|travis|circleci)|appveyor)})
end
end
view raw 018.rb hosted with ❤ by GitHub

…and replace it with this:

# in ruffle.gemspec
spec.files = ["Cargo.toml", "Cargo.lock", "src/lib.rs"]
view raw 019.rb hosted with ❤ by GitHub

We need to set the crate-type to "cdylib" in order to tell the Rust compiler that the output should be a shared library that can be used from other languages. As per the Rust docs on Linkage:

> --crate-type=cdylib, #[crate_type = "cdylib"] – A dynamic system library will be produced. This is used when compiling a dynamic library to be loaded from another language. This output type will create *.so files on Linux, *.dylib files on macOS, and *.dll files on Windows.

So next, we add crate-type to Cargo.toml:

# in Cargo.toml
[lib]
crate-type = ["cdylib"]
view raw 021.toml hosted with ❤ by GitHub

Now we can finally count on rake to build the crate:

$ rake compile
Successfully built RubyGem
Name: ruffle
Version: 0.1.0
File: ruffle.gem
Building native extensions. This could take a while…
Successfully installed ruffle-0.1.0
1 gem installed
view raw 022.sh hosted with ❤ by GitHub
💡 Click for additional details – what happens when we don’t set crate-type?

When we run $ rake compile, we get another error:

$ rake compile
# ...omitted

Compiling ruffle v0.1.0 (/Users/brian/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/gems/ruffle-0.1.0)
   Finished release [optimized] target(s) in 0.47s

Dynamic library not found for Rust extension (in /Users/brian/.rbenv/versions/3.0.0/lib/ruby/gems/3.0.0/extensions/x86_64-darwin-21/3.0.0/ruffle-0.1.0)

Make sure you set "crate-type" in Cargo.toml to "cdylib"

The error message came with instructions this time! Thanks, @ianks!


Defining the Ruffle module in Rust

While the crate is compiling now, our tests are still failing because we haven’t actually done anything with the Rust code. Since we’ll be creating the Ruby data structures on the Rust side, we’ll use a Ruby API to interface with Ruby internals. Enter rb-sys, Rust bindings for ruby that have been automatically generated using rust-bindgen.

Add it under your dependencies in Cargo.toml:

# in Cargo.toml
[dependencies]
rb-sys = { git = "https://github.com/ianks/rb-sys", tag = "v0.3.0" }
view raw 023.toml hosted with ❤ by GitHub

Next, we’ll just copy from rust_ruby_example/src/lib.rs, the example gem from the last post, except we’ll substitute “ruffle” wherever it says “rust_ruby_example” and “shuffle” wherever it says “reverse.”

// replace the contents of ruffle/src/lib.rs with the code below
use rb_sys::{
rb_define_module, rb_define_module_function, rb_string_value_cstr, rb_utf8_str_new, VALUE,
};
use std::ffi::{CStr, CString};
use std::os::raw::{c_char, c_long};
#[inline]
unsafe fn cstr_to_string(str: *const c_char) -> String {
CStr::from_ptr(str).to_string_lossy().into_owned()
}
#[no_mangle]
unsafe extern "C" fn pub_shuffle(_klass: VALUE, mut input: VALUE) -> VALUE {
let ruby_string = cstr_to_string(rb_string_value_cstr(&mut input));
let shuffled = ruby_string.chars().rev().collect::<String>();
let shuffled_cstring = CString::new(shuffled).unwrap();
let size = ruby_string.len() as c_long;
rb_utf8_str_new(shuffled_cstring.as_ptr(), size)
}
#[allow(non_snake_case)]
#[no_mangle]
pub extern "C" fn Init_ruffle() {
let name = CString::new("Ruffle").unwrap();
let shuffle = CString::new("shuffle").unwrap();
let callback = unsafe {
std::mem::transmute::<
unsafe extern "C" fn(VALUE, VALUE) -> VALUE,
unsafe extern "C" fn() -> VALUE,
>(pub_shuffle)
};
let klass = unsafe { rb_define_module(name.as_ptr()) };
unsafe { rb_define_module_function(klass, shuffle.as_ptr(), Some(callback), 1) }
}
view raw 024.rs hosted with ❤ by GitHub

Now we just remove the lib/ruffle.rb file:

$ rm lib/ruffle.rb
view raw 026.sh hosted with ❤ by GitHub
💡 Click for additional details – why delete lib/ruffle.rb?

The respond_to test won’t pass because Ruby’s require method will look in lib first and find the empty lib/ruffle.rb file before it searches available gems (you can read more about how Ruby’s require works here: https://ryanbigg.com/2017/11/how-require-loads-a-gem).

Here’s the proof:

$ rake compile spec

# ...omitted

Ruffle
  has a #shuffle method (FAILED - 1)

Failures:

  1) Ruffle has a #shuffle method
     Failure/Error: expect(Ruffle).to respond_to(:shuffle)
       expected Ruffle to respond to :shuffle
     # ./spec/ruffle_spec.rb:5:in `block (2 levels) in <top (required)>'

(Note that this leaves lib/ruffle/version.rb, which could be confusing for anyone reading your code.)

Now, let’s run rake spec:

$ rake spec
Ruffle
has a #shuffle method
Finished in 0.00205 seconds (files took 0.09782 seconds to load)
1 example, 0 failures
view raw 027.sh hosted with ❤ by GitHub

We’re green! ✅ And all with Rust code.

Implementing Ruffle#shuffle

Next, let’s add a test that describes the behavior of the method:

# in spec/ruffle_spec.rb
RSpec.describe Ruffle do
# …
it "shuffles the characters in a string" do
string = "Ruffle"
expect(Ruffle.shuffle(string)).to_not eq(string)
expect(Ruffle.shuffle(string)).to_not eq(string.reverse)
end
end
view raw 028.rb hosted with ❤ by GitHub

Technically, this could fail if the shuffled string randomly returns the original string. If this happens, you’ve won the random number generator lottery! Take a screenshot 📸

And when we run rake spec:

$ rake
Ruffle
has a #shuffle method
shuffles the characters in a string (FAILED – 1)
Failures:
1) Ruffle shuffles the characters in a string
Failure/Error: expect(Ruffle.shuffle(string)).to_not eq(string.reverse)
expected: value != "elffuR"
got: "elffuR"
view raw 029.sh hosted with ❤ by GitHub

Because we’ve simply copied over rust_ruby_example‘s pub_reverse code, it’s just reversing the string.

Let’s see if we can modify our pub_shuffle method in Rust to look like what we want. Right now we have:

// in lib.rs
#[no_mangle]
unsafe extern "C" fn pub_shuffle(_klass: VALUE, mut input: VALUE) -> VALUE {
let ruby_string = cstr_to_string(rb_string_value_cstr(&mut input));
let shuffled = ruby_string.chars().rev().collect::<String>();
let shuffled_cstring = CString::new(shuffled).unwrap();
let size = ruby_string.len() as c_long;
rb_utf8_str_new(shuffled_cstring.as_ptr(), size)
}
view raw 030.rs hosted with ❤ by GitHub

Instead of let shuffled = ruby_string.chars().rev().collect::<String>() on line 4, we would want something like let shuffled = ruby_string.chars().shuffle().collect::<String>(). Unfortunately, there is no such method implemented on Rust’s Iter struct.

As per this StackOverflow answer, you can use Rust’s rand::seq::SliceRandom trait to provide a shuffle method on Vecs. StackOverflow user Vladimir Matveev’s answer looks like this:

// from https://stackoverflow.com/a/26035435/1042144
use rand::thread_rng;
use rand::seq::SliceRandom;
fn main() {
let mut vec: Vec<u32> = (0..10).collect();
vec.shuffle(&mut thread_rng());
println!("{:?}", vec);
}
view raw 031.rs hosted with ❤ by GitHub

Shuffling a vector is a randomized operation, so we need a random number generator (RNG), and Rust doesn’t have an RNG in its standard library. The de facto standard RNG in Rust is rand. Let’s add it to our dependencies:

# in Cargo.toml
[dependencies]
#
rand = "0.8.4"
view raw 032.toml hosted with ❤ by GitHub

In the original code, the chars are reversed and collected into an owned String.

// in rust_ruby_example/src/lib.rs
let reversed = ruby_string.chars().rev().collect::<String>();
let reversed_cstring = CString::new(reversed).unwrap();
view raw 033.rs hosted with ❤ by GitHub

However, we need to actually mutate the Vec in place instead of using fancy functional Iter methods.

In rust_ruby_example the input goes from a Ruby VALUE type:

mut input: VALUE
view raw 034.rs hosted with ❤ by GitHub

To a CString type with this function call:

rb_string_value_cstr(&mut input)
view raw 035.rs hosted with ❤ by GitHub

Finally to a Rust String type with the final function call:

cstr_to_string(rb_string_value_cstr(&mut input))
view raw 036.rs hosted with ❤ by GitHub

We want to shuffle the characters of the string, so we’ll declare it as such and request the characters. Calling collect() charges you casts the value into the requested type signature if an Into trait is available for the type conversion:

let mut chars: Vec<char> = cstr_to_string(rb_string_value_cstr(&mut input))
.chars()
.collect();
view raw 037.rs hosted with ❤ by GitHub
💡 Additional notes about chars()

Note that Rust’s chars() method does not handle Unicode grapheme clusters (thanks, Wesley!). As per Rust’s documentation on the chars method:

Remember, chars might not match your intuition about characters:


    let y = "y̆";

    let mut chars = y.chars();

    assert_eq!(Some('y'), chars.next()); // not 'y̆'
    assert_eq!(Some('\u{0306}'), chars.next());

    assert_eq!(None, chars.next());
    

The more or less canonical way to handle this is to use the unicode-segmentation crate, as per this StackOverflow answer.

This is also a handy introduction to Unicode: The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)


Now we can bring the rand functionality into scope at the top of our file:

// at the top of lib.rs
use rand::seq::SliceRandom;
use rand::thread_rng;
view raw 038.rs hosted with ❤ by GitHub

And then call shuffle on the chars vec:

chars.shuffle(&mut thread_rng());
view raw 039.rs hosted with ❤ by GitHub

So your function should now look like this:

// in lib.rs
#[no_mangle]
unsafe extern "C" fn pub_shuffle(_klass: VALUE, mut input: VALUE) -> VALUE {
let mut chars: Vec<char> = cstr_to_string(rb_string_value_cstr(&mut input))
.chars()
.collect();
chars.shuffle(&mut thread_rng());
// …we'll add more code here
}
view raw 040.rs hosted with ❤ by GitHub

Now we need to convert the Vec<char> to a Rust String and store its length as a c_long type, which in this case is just a type alias for i64:

let shuffled: String = chars.iter().collect();
let size = shuffled.len() as c_long;
view raw 041.rs hosted with ❤ by GitHub

Then we construct a new CString from the shuffled Rust String:

let shuffled_cstring = CString::new(shuffled).unwrap();
view raw 042.rs hosted with ❤ by GitHub

And finally return the Ruby String:

rb_utf8_str_new(shuffled_cstring.as_ptr(), size)
view raw 043.rs hosted with ❤ by GitHub

…and with that, the final function looks like this:

// in lib.rs
#[no_mangle]
unsafe extern "C" fn pub_shuffle(_klass: VALUE, mut input: VALUE) -> VALUE {
let mut chars: Vec<char> = cstr_to_string(rb_string_value_cstr(&mut input))
.chars()
.collect();
chars.shuffle(&mut thread_rng());
let shuffled: String = chars.iter().collect();
let size = shuffled.len() as c_long;
let shuffled_cstring = CString::new(shuffled).unwrap();
rb_utf8_str_new(shuffled_cstring.as_ptr(), size)
}
view raw 044.rs hosted with ❤ by GitHub

And don’t forget to check your Init_ruffle method that actually initializes the Ruby Ruffle module and defines the method:

// in lib.rs (at the end)
#[allow(non_snake_case)]
#[no_mangle]
pub extern "C" fn Init_ruffle() {
let name = CString::new("Ruffle").unwrap();
let shuffle = CString::new("shuffle").unwrap();
let callback = unsafe {
std::mem::transmute::<
unsafe extern "C" fn(VALUE, VALUE) -> VALUE,
unsafe extern "C" fn() -> VALUE,
>(pub_shuffle)
};
let klass = unsafe { rb_define_module(name.as_ptr()) };
unsafe { rb_define_module_function(klass, shuffle.as_ptr(), Some(callback), 1) }
}
view raw 045.rs hosted with ❤ by GitHub

(It’s unmodified from our earlier template code.)

Now to recompile it and run the specs:

And we’re passing!


I’m sure there will be a lot of patterns emerging on how to organize the code once people start creating their own Rust gems – this is by no means a definitive one, just the one I copied from @ianks’s rust_ruby_example gem. I’m a novice as well! But hopefully you got as much out of reading this as I did out of writing it. And with @ianks’s pull request approved and passing CI, hopefully it’ll be only a matter of time before everyone gets to play with the new functionality!

The commit messages are pretty bad because it was a spike for me, but you can find all the code here: https://github.com/briankung/ruffle

6 responses to “Writing a Rust gem from scratch”

  1. […] If this post tickled your fancy, check out the follow-up post: Writing a Rust gem from scratch […]

  2. RFC: what’s a better way to do this?

    instead of using system, use backticks to execute — it eats STDOUT and returns it as a value, so you can do something like:

    gem_exists = `gem list -i ^ruffle$` == 'true'
    
    1. I’ll change that right away, thanks Zach!

  3. Note that Rust’s chars() method does not handle Unicode

    Rust very much does handle Unicode, I think you mean that chars does not yield grapheme clusters, it only yields individual Unicode code points.

    Did you ever find out the reason for the mem::transmute? It’s a function that very often misued/used in a way that introduces undefined behaviour. As the documentation notes:

    transmute is incredibly unsafe. There are a vast number of ways to cause undefined behavior with this function. transmute should be the absolute last resort.

    1. I think you mean that chars does not yield grapheme clusters

      You’re right, that’s what I meant! I’ll update the wording to reflect that.

      Did you ever find out the reason for the mem::transmute?

      I have not. I haven’t had a lot of time to do much else other than polishing the article up. I’ve sent a tweet to Ian on Twitter asking what it’s for.

      After staring at the C header file where rb_define_module_function is defined – I don’t know C 😰 – I think it’s necessary because Rust won’t let you pass a function pointer with arbitrary arity, but the C code just assumes that you can. Note that the last argument in rb_define_module_function is an arity indicator. So the transmutation is just ceremony to get a function pointer – any function pointer – past Rust’s type system. That’s my guess, anyway.

      EDIT – Seems right! https://twitter.com/_ianks/status/1489419634168184834

      1. Right looks like ANYARGS gets repeatedly expanded to expect a function pointer with arity number of arguments.

        https://cs.github.com/ruby/ruby/blob/d66e7ec77b0067b113e1b9f584e7f5f741d6cd78/include/ruby/internal/anyargs.h#L248

        As Ian’s tweet mentions, bindgen declares rb_define_module_function as:

        extern "C" {
            pub fn rb_define_module_function(
                arg1: VALUE,
                arg2: *const ::libc::c_char,
                arg3: ::core::option::Option<unsafe extern "C" fn() -> VALUE>,
                arg4: ::libc::c_int,
            );
        }
        

        so Rust expects a pointer to a function that take no arguments and returns a Ruby value, hence the need to transmute.

        Rutie, mentioned in the tweet always uses the argc, argv form of the callback (arity -1) to that transmuting isn’t necessary:

        https://github.com/danielpclark/rutie/blob/cba311cbb5873ef42ad627081f2dec04feab9a51/src/binding/module.rs#L22-L28

Leave a Reply to Zach L Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s