Jeroen Ooms

## Abstract / TLDR

[R-universe](https://r-universe.dev) now provides MacOS arm64 binaries
for all R packages. This means that MacOS users on Apple Silicon
hardware (aka M1/M2/M3) can install the very latest builds of any R
package without the need for any compilation:

::: {.highlight}
``` {.chroma tabindex="0"}
install.packages('arrow',  repos = c('https://apache.r-universe.dev', 'https://cloud.r-project.org'))
```
:::

R-universe uses cross-compiling for arm64 binaries, though this should
not make much of a difference for package authors and R users. Packages
with C/C++/Fortran/Rust code are all supported.

## Why cross compiling

Because GitHub Actions currently does not offer arm64 runners for OSS
projects, the arm64 binaries are cross-compiled on the MacOS intel
runners. The cross build environment is set up to mimic a native arm64
machine, such that most R packages do not need any modification to work.
We found only a small number of packages with a buggy configure script
that may need a fix to allow cross-compilation.

The r-universe workflow only builds arm64 binaries when needed, i.e. for
packages that include compiled code (C/C++/Fortran/Rust). Packages that
do not include any compiled code are portable by design, so for these
packages the binaries built for MacOS on intel are served both in the
x86_64 and arm64 cranlike repositories, without the need for an
additional cross compile step.

Just like CRAN, the r-universe package homepage shows a link to the
`r-4.3-x86_64` and `r-4.3-arm64` binaries. Packages without compiled
code have a `r-4.3-any` binary which is used for either architecture.

::: {.box}
<figure itemprop="associatedMedia" itemscope=""
itemtype="http://schema.org/ImageObject">
<div class="img">
<img src="example-arrow.png" itemprop="thumbnail"
alt="Screenshot of r-universe apache arrow homepage" />
</div>
<a href="example-arrow.png" itemprop="contentUrl"></a>
</figure>
:::

To have a look at the build logs, click on the little apple icon next to
these links. Alternatively, you can use the `/buildlog` shortlink, for
example `https://apache.r-universe.dev/arrow/buildlog` will take you to
the latest arrow builds.

On this page you can find the arm64 build log specifically by expanding
the `r-release-macos` job and then under the section "Cross Compile for
MacOS ARM64". If this section is disabled, it means it was skipped
because this package does not have compiled code, and does not need
cross compilation.

::: {.box}
<figure itemprop="associatedMedia" itemscope=""
itemtype="http://schema.org/ImageObject">
<div class="img">
<img src="macos-cross.png" itemprop="thumbnail"
alt="Screenshot of GitHub Actions output" />
</div>
<a href="macos-cross.png" itemprop="contentUrl"></a>
</figure>
:::

## Some technical details

For those interested how the cross compilation is set up, here are the
main ingredients:

- The standard MacOS Xcode toolchains are used to cross compile C/C++
  code by passing the `-arch arm64` flag to clang and clang++.
- The [universal gfortran 12.2](https://mac.r-project.org/tools/)
  version from CRAN (thanks to Simon Urbanek) is used to cross compile
  fortran code, also by passing `gfortran -arch arm64`.
- The same collection of [system
  libs](https://mac.r-project.org/bin/darwin20/arm64/) used by CRAN is
  preinstalled in the build environment.
- R packages with a configure script are built with
  `--configure-args="--build=x86_64-apple-darwin20 --host=aarch64-apple-darwin20"`.
  These flags are [needed by autoconf
  scripts](https://www.gnu.org/software/autoconf/manual/autoconf-2.68/html_node/Specifying-Target-Triplets.html),
  but other packages can use them as well.
- The r-universe [macos-cross
  workflow](https://github.com/r-universe-org/build-and-check/blob/v1/macos-cross/action.yml)
  overrides some more files and variables to target arm64.
- We put some shell
  [shims](https://github.com/r-universe-org/prepare-macos/tree/master/shims)
  on the PATH to help packages that shell out to `uname` or `arch` to
  determine the architecture.
- A clever [cargo
  shim](https://github.com/r-universe-org/prepare-macos/blob/master/shims/cargo.sh)
  is used to override the default cargo build target to
  `aarch64-apple-darwin` and copy outputs to the expected directory
  after the build.
- Packages are [built with strict
  linking](https://stat.ethz.ch/pipermail/r-sig-mac/2024-January/014912.html)
  (without the `-undefined dynamic_lookup` flag).

With this setup, almost any R package can be built in the cross
environment exactly the same way they do on normal arm64 hardware. But
if your package does not work and you need some help fixing it, please
feel free to [open an
issue](https://github.com/r-universe-org/help/issues).

## Universal binaries

Finally, some R packages download precompiled binaries for libraries too
big or complicated to build on the fly. It is safest to distribute such
libraries in the form of [universal
binaries](https://en.wikipedia.org/wiki/Universal_binary) which contain
both the x86_64 and arm64 libs. This way your download script does not
need to make any guesses about the target architecture it is building
for: the same libs can be linked on either target.

Creating a universal binary can be done for both static and dynamic
libraries and is really easy. If you have a `x86_64` and `arm64` version
of `libfoo.a` you can glue them together with lipo:

::: {.highlight}
``` {.chroma tabindex="0"}
lipo -create x86_64/libfoo.a arm64/libfoo.a -output universal/libfoo.a
```
:::

And this new `libfoo.a` can be used when building either for intel or
arm. Again if you need any help with this feel free to reach out.


Abstract / TLDR   R-universe now provides MacOS arm64 binaries for all R packages.

R-universe now builds MacOS ARM64 binaries for use on Apple Silicon (aka M1/M2/M3) systems

Georgia State University

Andrew Heiss

GSU uses Microsoft's Office365 for e-mail, which is fine. My previous
institutions---Duke and BYU---both use it too, and it's pretty standard.
GSU also enforces 2-factor authentication (2FA) with Duo, which is also
fine. Everybody should use some sort of 2FA for all their important
accounts!

However, for whatever reason, GSU's version of Duo's 2FA doesn't allow
you to generate app-specific passwords for things like e-mail. Instead,
any e-mail client I use has to have support for Microsoft's special
[Modern
Authentication](https://docs.microsoft.com/en-us/office365/enterprise/office-365-client-support-modern-authentication)
system, which opens up a popup window to handle the 2FA and logging in
and everything. The issue with this is that very few e-mail clients
support Modern Authentication. In the macOS world, the only program that
supports it is Apple Mail. That's all.

This means I've had to move away from my favorite e-mail client ever:
[Airmail](https://airmailapp.com/). Airmail is fast, looks nice, and has
great search features. Most importantly for me, though, is that it let
you write e-mails in Markdown and then converted the Markdown text to
HTML when you clicked send. This is the coolest thing ever if you use
Markdown everywhere normally, but it's even better when teaching
code-heavy classes. I could respond to student questions by typing stuff
like:

::: {#cb1 .sourceCode style="background: #f1f3f5;"}
```` {.sourceCode .markdown .code-with-copy}
Oh, I see! You have an error in the 3rd line there. Try changing it to:

```r
ggplot(mtcars, aes(x = wt, y = mpg)) +
  geom_point() +
  theme_bw()
```
````
:::

...and Airmail would convert that to nicely formatted HTML. So
convenient!

Apple Mail can't do this.

However, through the magic of macOS services, Bash scripting, and
AppleScript, I've found a way to convert Markdown text to richly
formatted text, and it's delightful!

Here's how to do it.

::: {#macos-services .section .level2 .page-columns .page-full}
## macOS services {#macos-services .anchored anchor-id="macos-services"}

macOS comes with Automator, a program that lets you create workflows for
repeated tasks. One kind of workflow is called a Service (or Quick
Action), which can take text (or a file), do stuff to it, and spit out
new text. Here's a super basic example that takes selected text,
converts it to uppercase with the `tr` bash command, copies it to the
system-wide clipboard using the `pbcopy` shell script, gets the contents
of the clipboard, and then replaces the selected text:

::: {.quarto-figure .quarto-figure-center .page-columns .page-full}
<figure class="figure page-columns page-full">
<p><img
src="https://www.andrewheiss.com/blog/2019/10/09/convert-md-rtf-macos-services/make-uppercase.png"
class="img-fluid figure-img" /></p>
<figcaption>Workflow to make text uppercase</figcaption>
</figure>
:::

If you save this as a Quick Action, macOS will put it in a folder named
`~/Library/Services`. Once it's there, it'll be accessible in any
program that lets you type, like TextEdit or Mail. Type some text in
TextEdit, select it, and go to the TextEdit → Services menu. You should
see the "Make uppercase" service. If you click on it, your text will be
converted to uppercase. Magic.

::: {.quarto-figure .quarto-figure-center .page-columns .page-full}
<figure class="figure page-columns page-full">
<p><img
src="https://www.andrewheiss.com/blog/2019/10/09/convert-md-rtf-macos-services/services-menu.png"
class="img-fluid figure-img" /></p>
<figcaption>Services menu in TextEdit</figcaption>
</figure>
:::

You can make these easier to run by assigning them keyboard shortcuts.
Go to System Preferences → Keyboard → Shortcuts → Services, scroll down
the list until you find the "Make uppercase" service, and add a shortcut
for it. Now you can convert text to upper case in any application by
selecting it and pressing the keyboard shortcut. Super magic.

::: {.quarto-figure .quarto-figure-center .page-columns .page-full}
<figure class="figure page-columns page-full">
<p><img
src="https://www.andrewheiss.com/blog/2019/10/09/convert-md-rtf-macos-services/add-shortcut.png"
class="img-fluid figure-img" /></p>
<figcaption>Assigning a keyboard shortcut to a service</figcaption>
</figure>
:::
:::

::: {#markdown-to-rtf-basic .section .level2 .page-columns .page-full}
## Markdown to RTF, basic {#markdown-to-rtf-basic .anchored anchor-id="markdown-to-rtf-basic"}

Rather than converting text to uppercase, we can make a service that
pipes Markdown text through [pandoc](https://pandoc.org), converts it to
nicely styled RTF, and replaces the selected text with the nicely styled
text.

Make a new Quick Action in Automator that looks like this (I named it
**md2rtf**):

::: {.quarto-figure .quarto-figure-center .page-columns .page-full}
<figure class="figure page-columns page-full">
<p><img
src="https://www.andrewheiss.com/blog/2019/10/09/convert-md-rtf-macos-services/workflow-simple.png"
class="img-fluid figure-img" /></p>
<figcaption>Simple md2rtf workflow</figcaption>
</figure>
:::

The shell script should look like this:

::: {#cb2 .sourceCode style="background: #f1f3f5;"}
``` {.sourceCode .sh .code-with-copy}
# !/bin/bash
export LC_CTYPE=UTF-8
/usr/local/bin/pandoc -t rtf -s | pbcopy
```
:::

This will run your text through pandoc, convert it to RTF, and copy the
results to the clipboard. The "Get Contents of Clipboard" will then grab
the formatted text from the clipboard and replace the selected text with
it.

Watch it in action here:

With this service, I can type in Markdown in Mail, convert it all to
rich text, and then send, which is really convenient!
:::

::: {#markdown-to-rtf-with-syntax-highlighting .section .level2 .page-columns .page-full}
## Markdown to RTF, with syntax highlighting {#markdown-to-rtf-with-syntax-highlighting .anchored anchor-id="markdown-to-rtf-with-syntax-highlighting"}

However, it's not quite perfect. RTF doesn't support syntax
highlighting, so if I convert any code, it'll format it in monospaced
Courier font (which is good!) that is just plain black (which is less
good!). HTML output does support syntax highlighting, though, so it'd be
nice if there was a way to take Markdown text and replace it with
converted HTML.

Just changing the pandoc script to
`/usr/local/bin/pandoc -t html -s | pbcopy` won't work, though. It'll
convert the file to HTML, as expected, but it'll replace your text with
all the HTML tags instead of rendered HTML, which is less than ideal.

::: {.quarto-figure .quarto-figure-center .page-columns .page-full}
<figure class="figure page-columns page-full">
<p><img
src="https://www.andrewheiss.com/blog/2019/10/09/convert-md-rtf-macos-services/wrong-html.png"
class="img-fluid figure-img" /></p>
<figcaption>Incorrect unrendered raw HTML</figcaption>
</figure>
:::

So instead, we need to convert to HTML, somehow render that HTML to rich
text, and then replace the text with that instead. Fortunately someone
asked a [similar question on StackOverflow in
2012](https://stackoverflow.com/a/11089226), and there's a solution we
can adapt! We basically convert HTML to raw hex code, then convert the
hex code to HTML with AppleScript, which renders the HTML correctly. It
seems (and is) convoluted, but it works!

Change the shell script in the Automator workflow to this:

::: {#cb3 .sourceCode style="background: #f1f3f5;"}
``` {.sourceCode .sh .code-with-copy}
# !/bin/bash
export LC_CTYPE=UTF-8
hex=`/usr/local/bin/pandoc -t html -s --highlight-style pygments | hexdump -ve '1/1 "%.2x"'`
osascript -e "set the clipboard to «data HTML${hex}»"
```
:::

::: {.quarto-figure .quarto-figure-center .page-columns .page-full}
<figure class="figure page-columns page-full">
<p><img
src="https://www.andrewheiss.com/blog/2019/10/09/convert-md-rtf-macos-services/workflow-better.png"
class="img-fluid figure-img" /></p>
<figcaption>Better md2rtf workflow</figcaption>
</figure>
:::

This will take the selected text, convert it to HTML, convert the raw
HTML to hex codes, convert the hex code to *rendered* HTML, and replace
the selected text with that.

Here's what it looks like:

This is almost perfect! The only minor issue is that the non-code text
switched from Helvetica (TextEdit's and Apple Mail's default) to Times
New Roman, which isn't great. It'd be fantastic if the converted HTML
used Helvetica instead of Times.

Fortunately there's a way to fix that. The text is getting converted to
Times because the rendered HTML defaults to Times in the absence of any
CSS styles telling it to be something else. If we can insert some custom
CSS into the converted HTML file with pandoc, we should be able to get
the correct font.

There's an argument for pandoc that lets you insert files into the head
of the HTML, `-H`. (There's also a `--css` argument, but [it doesn't
play well with standalone HTML
files](https://devilgate.org/blog/2012/07/02/tip-using-pandoc-to-create-truly-standalone-html-files/),
so it's easier to insert stuff directly into the converted HTML). Create
an HTML file somewhere on your computer with this:

::: {#cb4 .sourceCode style="background: #f1f3f5;"}
``` {.sourceCode .html .code-with-copy}
<style type="text/css">
body {
    font-family: Helvetica, sans-serif;
}
</style>
```
:::

(This isn't raw CSS---it's CSS wrapped in HTML. We have to do that
because pandoc will take that whole file and insert it as HTML in the
converted document, so we have to treat it as HTML.)

Change your Automator workflow one last time so that it injects the
custom CSS:

::: {#cb5 .sourceCode style="background: #f1f3f5;"}
``` {.sourceCode .sh .code-with-copy}
# !/bin/bash
export LC_CTYPE=UTF-8
hex=`/usr/local/bin/pandoc -t html -s --highlight-style pygments -H ~/path/to/your/css/thing/md2rtf_styles.html | hexdump -ve '1/1 "%.2x"'`
osascript -e "set the clipboard to «data HTML${hex}»"
```
:::

::: {.quarto-figure .quarto-figure-center .page-columns .page-full}
<figure class="figure page-columns page-full">
<p><img
src="https://www.andrewheiss.com/blog/2019/10/09/convert-md-rtf-macos-services/workflow-best.png"
class="img-fluid figure-img" /></p>
<figcaption>Best md2rtf workflow</figcaption>
</figure>
:::

With that addition, the workflow will now take your selected text,
convert it to HTML that is styled with Helvetica, convert *that* to hex
code, convert *that* to rendered HTML, and finally replace your text
with impeccable style:

I've added **⌘⌥\^⇧P** as the shortcut for this (so I essentially mash
down the whole bottom left corner of my keyboard and hit P), and it
makes using Apple Mail with Markdown and code quite convenient!
:::

::: {#quarto-appendix .default}
::: {.section .quarto-appendix-contents}
## Citation {#citation .anchored .quarto-appendix-heading}

<div>

::: {.quarto-appendix-secondary-label}
BibTeX citation:
:::

``` {.sourceCode .code-with-copy .quarto-appendix-bibtex}
@online{heiss2019,
  author = {Heiss, Andrew},
  title = {Convert {Markdown} to Rich Text (with Syntax Highlighting!)
    in Any {macOS} App},
  pages = {undefined},
  date = {2019-10-09},
  url = {https://www.andrewheiss.com/blog/2019/10/09/convert-md-rtf-macos-services},
  doi = {10.59350/y07h7-cn637},
  langid = {en}
}
```

::: {.quarto-appendix-secondary-label}
For attribution, please cite this work as:
:::

::: {#ref-heiss2019 .csl-entry .quarto-appendix-citeas}
Heiss, Andrew. 2019. "Convert Markdown to Rich Text (with Syntax
Highlighting!) in Any macOS App." October 9, 2019.
<https://doi.org/10.59350/y07h7-cn637>.
:::

</div>
:::
:::


GSU uses Microsoft’s Office365 for e-mail, which is fine. My previous institutions—Duke and BYU—both use it too, and it’s pretty standard. GSU also enforces 2-factor authentication (2FA) with Duo, which is also fine. Everybody should use some sort of 2FA for all their important accounts! However, for whatever reason, GSU’s version of Duo’s 2FA doesn’t allow you to generate app-specific passwords for things like e-mail.

Rogue Scholar Posts

R-universe now builds MacOS ARM64 binaries for use on Apple Silicon (aka M1/M2/M3) systems

Convert Markdown to rich text (with syntax highlighting!) in any macOS app