Permutation tests for heterogeneity comparisons in presence of categorical variables with application to university evaluation

In social sciences researchers often meet the problem of determining if the distribution of a categorical variable is more concentrated in population X1 than in population X2. For example the effectiveness of two different PhD programs can be evaluated in terms of the heterogeneity of the set of job opportunities. The nonparametric solution of this problem has similarities to that of permutation testing for stochastic dominance on ordered categorical variables, i.e. testing under order restrictions. If ordering of probability parameters in H0 is unknown and it has to be estimated by sampling data, only approximate nonparametric solutions are possible within the permutation approach. Main properties of test solutions and some Monte Carlo simulations in order to evaluate the tests’ behaviour under H0 and H1, will be presented. A real problem concerned with University evaluation is also discussed.