Mapping the average active galactic nucleus (AGN) luminosity across galaxy populations and over time reveals important clues regarding the interplay between supermassive black hole and galaxy growth. This paper presents the demography, mean power, and cosmic evolution of radio AGN across star- forming galaxies (SFGs) of different stellar masses (ℳ*). We exploit deep VLA-COSMOS 3 GHz data to build the rest-frame 1.4 GHz AGN luminosity functions at 0.1 ≤ z ≤ 4.5 hosted in SFGs. Splitting the AGN luminosity function into different ℳ* bins reveals that, at all redshifts, radio AGN are both more frequent and more luminous in higher ℳ* than in lower ℳ* galaxies. The cumulative kinetic luminosity density exerted by radio AGN in SFGs peaks at z ∼ 2, and it is mostly driven by galaxies with 10.5 ≤ log(ℳ*/ ℳ⊙) < 11. Averaging the cumulative radio AGN activity across all SFGs at each (ℳ*, z) results in a `radio-AGN main sequence' that links the time-averaged radio-AGN power ⟨L1.4AGN⟩ and galaxy stellar mass, in the form: log ⟨[L1.4AGN/ W Hz−1]⟩ = (20.97 ± 0.16) + (2.51 ± 0.34)⋅ log(1+z) + (1.41 ± 0.09)⋅(log[ℳ*/ℳ⊙] - 10). The super-linear dependence on ℳ*, at fixed redshift, suggests enhanced radio-AGN activity in more massive SFGs as compared to star formation. We ascribe this enhancement to both a higher radio AGN duty cycle and a brighter radio-AGN phase in more massive SFGs. A remarkably consistent ℳ* dependence is seen for the evolving X-ray AGN population in SFGs. This similarity is interpreted as possibly driven by secular cold gas accretion fuelling both radio and X-ray AGN activity in a similar fashion over the galaxy's lifetime.