We don't currently support changing the gain of the FEM to lower power with an API. This is not supported in the Nordic driver. The FEM has two calibrated gain settings - one that achieves 20 dBm output and one that achieves 10 dBm output (both with 0 dBm input). However, you can change the output power level as discussed in the following FAQ.

 Can I lower the TX Power on the BL5340 from the maximum set by the FW? (ezurio.com)

The power level can lowered at compile time by changing CONFIG_BT_CTLR_TX_PWR_ANTENNA from 20 to x. Power is split into two pieces (nrf53 + FEM).  Most of the granularity in power steps of the nRF53 is used obtain the output required for RF compliance. Therefore, the granularity of any further reductions in output power is limited.

For example, for TX Power = 12, this value is specified via CONFIG_BT_CTLR_TX_PWR_ANTENNA and the driver will try to get as close as it can.

The nRF5340 SoC output power has granularity of 0, -1, -2, -3, -4, -5, -6, -7, -8, -12, -16, -20, -40. However, 0 to -8 may already be used to achieve compliance for 20 dBm output at the antenna.