We investigate the environments of both X-ray and radio-loud AGN within the UKIDSS Ultra-deep Survey (UDS) using deep infrared selection to sample the galaxy density field in the redshift range 1.0 <= z ~ 10^13 M_sun. A closer look at the small-scale environments of the AGN reveals that the neighbouring galaxies of radio-loud AGN have U-B colours more skewed towards the `green-valley' and the red sequence, whereas the neighbours of X-ray AGN show no difference to the general galaxy population. This suggests that although both AGN types live in overdense environments, the radio-loud AGN may be preferentially located in more evolved cluster cores, in a similar environment to low-powered radio AGN in the local Universe.