According to templates/cassiopeia/joomla.asset.json the file system/joomla-fontawesome.min.css is loaded

https://github.com/joomla/joomla-cms/blob/816cd8013012aa5faf21c53636a0a14f17caa64b/templates/cassiopeia/joomla.asset.json#L104C1-L108C6

The compiled file can be found in /media/system/css/joomla-fontawesome.min.css

in this file I can find a reference to the font-face... at least on my machine.

:root, :host {
    --fa-style-family-classic: "Font Awesome 6 Free";
    --fa-font-regular: normal 400 1em/1 "Font Awesome 6 Free"
}

@font-face {
    font-family: "Font Awesome 6 Free";
    font-style: normal;
    font-weight: 400;
    font-display: block;
    src: url(../../../media/vendor/fontawesome-free/webfonts/fa-regular-400.woff2?v=2d89b4) format("woff2"), url(../../../media/vendor/fontawesome-free/webfonts/fa-regular-400.ttf?v=370dd5) format("truetype")
}

.far, .fa-regular {
    font-weight: 400
}

:root, :host {
    --fa-style-family-classic: "Font Awesome 6 Free";
    --fa-font-solid: normal 900 1em/1 "Font Awesome 6 Free"
}

@font-face {
    font-family: "Font Awesome 6 Free";
    font-style: normal;
    font-weight: 900;
    font-display: block;
    src: url(../../../media/vendor/fontawesome-free/webfonts/fa-solid-900.woff2?v=237f4a) format("woff2"), url(../../../media/vendor/fontawesome-free/webfonts/fa-solid-900.ttf?v=adec7d) format("truetype")
}

.fas, [class^=icon-], [class*=\ icon-], .fa-solid {
    font-weight: 900
}

:root, :host {
    --fa-style-family-brands: "Font Awesome 6 Brands";
    --fa-font-brands: normal 400 1em/1 "Font Awesome 6 Brands"
}

@font-face {
    font-family: "Font Awesome 6 Brands";
    font-style: normal;
    font-weight: 400;
    font-display: block;
    src: url(../../../media/vendor/fontawesome-free/webfonts/fa-brands-400.woff2?v=715d59) format("woff2"), url(../../../media/vendor/fontawesome-free/webfonts/fa-brands-400.ttf?v=56c8d8) format("truetype")
}

this is what I see on my machine.

In my component I'm building I want to use the vanilla fontawsome.css from the media/vendor folder. And right now it cannot be used as there is no font files loaded in it.

That is why I am questioning if it is meant to be used as standalone or should be compiled from the scss files.

After installation of Joomla the fonts (ttf, woff2) can be found in media/vendor/font-awesome-free/webfonts.
If they are not present in yours, then there might be an issue with your installation

Yeah, that a bug. default fontawesome.css does not include woff2.
There was a fix for j4 #31364 , but for j5 it probably need a litle more work.

Joomla have to provide fontawesome/all.css (which include woff files) instead of fontawesome/fontawesome.css

Note: you can use

In your template, the same as joomla cassiopeia do, which also includes some classes for core icon-.

I think, the proper way here would be to build what you need based on the supplied scss files:
https://docs.fontawesome.com/web/use-with/scss