2016 Grant Allocation Review
The following review was completed by Peter Shaffery, PHD Candidate in Applied Mathematics, as part of his membership on the UGGS Finance Committee under the direction of the Vice President of Finance. The full report is attached at the bottom of this page.
This plot gives a running mean of the bias towards or against each department which has submitted applications for UGGS travel grants. Bias was measured as the difference between proportion of grants given and proportion of applications received. Each colored line represents one department. Labels were used to indicate departments which, as of Fall 2016, had a mean bias that was greater than 1.5 standard deviations away from the mean taken across all departments.
Please note that lines starting far from 0 and sloping towards it indicate a one or two time large bias, followed by more regular behavior. Most outlier departments are only outliers due to an anomalous fluctuation in one or two years. These fluctuations are not statistically significant (as determined by a one-way F test to check for significant differences between departments in terms of the year-to-year bias). Data and analysis scripts can be made available upon request."
Here are some more specific details or explanations that you can use as you want:
"Bias" was measured as the difference between:
(1) the percentage of grants awarded to a department (relative to all grants awarded)
(2) the percentage of applications received from that department (relative to all applications received).
For example: if in total we received 100 applications and awarded 10 grants, then the measured bias for a department which submitted 20 applications and received 1 grant would be calculated: (1/10) - (20/100) = .1 - .2 = -.1
For every department which submitted grant applications we calculated bias for each year from 2012-2016. From this we then calculated a running time-average of these biases. The plot here shows this running mean up to year. This means that for year 2014 the plot provides the average of the biases in 2012, 2013, 2014. For year 2015 the plot provides the same average but also including the bias from the year 2015.
The running mean is a better way of representing this data over time compared to showing the actual biases in each year for a department. Due to chance, in a single year there will almost certainly be some department which is over or under awarded grants relative to how many applications it submitted. However if the awards process is truly unbiased, then these fluctuations should not be frequent, and for a single department should be negative about as often as they are positive. Therefore, if there is no "true" bias towards any department, these fluctuations should cancel out and the mean bias for a department should go to 0 as time passes.
To determine if these fluctuations were significant I first conducted two-way ANOVAs for the "applications received" and "grants awarded" data sets, checking if there were significant differences between apps or grants across departments and across years. Both of these tests indicated that the fluctuations were significant; that is that both the department and the year had a statistically significant effect on how many grants were awarded or applications were received.