A radio button is correct.

On this screen you can only do one thing - make a language default and you can only make one language default to its a radio

Whether I can choose between different actions or have only one possible action for selected items makes no difference.
This can happen also in other views, depending on access rights.

In menus, where we can set only one menuItem as "home", we also have checkboxes. As there are only two states possible "checked" or "not checked" and no alternatives are present, I think it is better when all screens are equal.

if it is possible to select multiple items then it must be a check box
If it is only possible to select one item then it must be a radio

Remember that when you select a radio then it unselects any previous selection.

Ever since the first edition of Inside Macintosh in 1984, the rule has been the same for when to use checkboxes versus radio buttons. All subsequent GUI standards and the official W3C Web standards have included the same definition of these two controls:

1. Radio buttons are used when there is a list of two or more options that are mutually exclusive and the user must select exactly one choice. In other words, clicking a non-selected radio button will deselect whatever other button was previously selected in the list.

2. Checkboxes are used when there are lists of options and the user may select any number of choices, including zero, one, or several. In other words, each checkbox is independent of all other checkboxes in the list, so checking one box doesn't uncheck the others.

3. A stand-alone checkbox is used for a single option that the user can turn on or off.

Source NN Group

Radio and checkboxes also have inherent semantic meanings

Radio button is the correct usage as only one should be selected to be the default. Otherwise, you will run into this issue:
#24591

In menus, where we can set only one menuItem as "home", we also have checkboxes. As there are only two states possible "checked" or "not checked" and no alternatives are present, I think it is better when all screens are equal.

Because there are other actions that will perform on multiple items. In this case, there is only one action Default on one item.