Can I use the median p-value as a representation of the statistical significance of finding an category?
No! The median p-value is solely intended to be used for ranking the categories according to their biological association. It is not meant to quantify the statistical significance of the results and is in fact incorrect if used that way. The p-value is computed by using a null distribution that cannot provide an unbiased estimate of the categorys statistical significance.