aspencer comments on Update on Envision: progress thus far and next steps - Effective Altruism Forum

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (3)

You are viewing a single comment's thread. Show more comments above.

Comment author: aspencer 07 October 2017 05:29:40PM 0 points [-]

Our conference does have an application process. To provide accommodation for participants, travel reimbursement, and events that allow participants to engage with speakers we can only have a limited number of participants. Right now, we seek to maximize the impact that the conference has by selecting participants who are likely to be leaders in the future. We've narrowed down criteria we believe are indicative of becoming future leaders.

That said, I do agree with your point and I worry about our ability to accurately predict who will become a leader in the future. There are many possible paths to becoming a leader in the future, and what you do in college may not reflect well what you'll do in the future. Beyond that, applications are subjective and we likely have a high false negative rate. Some people are better at writing applications, and skill in writing applications doesn't imply (I think) the applicant will be successful in the future.

We've considered doing randomized evaluation, but haven't yet because we don't know of a rigorous way to measure how we did selecting participants that will have an impact in the future. An impact in the future would, by construction, have to be measured years after the conference. We'd have to track randomized applicants and applicants we selected and have a way to measure their impact, which would be challenging if not infeasible. Right now, the best heuristic we have for identifying applicants that will be leaders in the future is by looking at what they're doing now.

The act of having an application also provides self selection for applicants; many people have communicated to us that completing the application is a barrier to entry. The application itself selects for a certain level of interest in the conference (the interest required to complete the application).

We like to believe that this approach is better than random, but we do admit that it may not be.