19

Gregory_Lewis comments on MIRI Update and Fundraising Case - Effective Altruism Forum

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (16)

You are viewing a single comment's thread. Show more comments above.

Comment author: Gregory_Lewis 22 October 2016 06:58:07PM *  4 points [-]

Many thanks for the reply, Rob, and apologies for missing the AMA - although this discussion may work better in this thread anyway.

Respectfully, my reading of the Open Phil report suggests it is more broadly adverse than you suggest: in broad strokes, the worries are 1) That the research MIRI is undertaking probably isn't that helpful at improving AI risk; and 2) The research output MIRI has made along these lines is in any case unimpressive. I am sympathetic to both lines of criticism, but I am more worried by the latter than the former: AI risk is famously recondrite, thus diversity of approaches seems desirable.

Some elements of Open Phil's remarks on the latter concern seem harsh to me - in particular the remark that the suite of papers presented would be equivalent to 1-3 year's work from an unsupervised grad student is inapposite given selection, and especially given the heartening progress of papers being presented at UAI (although one of these is by Armstrong, who I gather is principally supported by FHI).

Yet others are frankly concerning. It is worrying that many of the papers produced by MIRI were considered unimpressive. It is even more worrying that despite the considerable efforts Open Phil made to review MIRI's efficacy - comissioning academics to review, having someone spend a hundred hours looking at them, etc. - they remain unconvinced of the quality of your work. That they emphasize fairly research-independent considerations in offering a limited grant (e.g. involvement in review process, germinating SPARC, hedging against uncertainty of approaches) is hardly a ringing endorsement; that they expressly benchmark MIRI's research quality as less than a higher end academic grantee likewise; comparison to other grants Open Phil have made in the AI space (e.g. 1.1M to FLI, 5.5M for a new center at UC Berkeley) even more so.

It has been remarked on this forum before MIRI is a challenging organisation to evaluate as the output (technical research in computer science) is opaque to most without a particular quantitative background. MIRI's predictions and responses to Open Phil implies a more extreme position: even domain experts are unlikely to appreciate the value of MIRI's work without a considerable back-and-forth with MIRI itself. I confess scepticism at this degree of inferential distance, particularly given the Open Phil staff involved in this report involved several people who previously worked with MIRI.

I accept MIRI may not be targetting conventional metrics of research success (e.g. academic publications). Yet across most proxy indicators (e.g. industry involvement, academic endorsement, collaboration) for MIRI 'doing good research', the evidence remains pretty thin on the ground - and, as covered above, direct assessment of research quality by domain experts is mixed at best. I look forward to the balance of evidence shifting favourably: the new conference papers are promising, ditto the buzz around logical induction (although I note the blogging is by people already in MIRI's sphere of influence/former staff, and MIRI's previous 'blockbuster result' in decision theory has thus far underwhelmed). Yet this hope, alongside the earnest assurances of MIRI that - if only experts gave them the time - they would be persuaded of their value, is not a promissory note that easily justifies an organisation with a turnover of $2M/year, nor fundraising for over a million dollars more.

Comment author: Gregory_Lewis 27 October 2016 07:22:42PM 7 points [-]

I take this opportunity to note I have made an even-odds bet with Carl Shulman for $1000, donated to the charity of the winner's choice over whether Open Phil's next review of MIRI has a more favourable evaluation of their research.

Comment author: Gregory_Lewis 09 November 2017 01:22:42AM *  8 points [-]

I am wiser, albeit poorer: the bet resolved in Carl's favour. I will edit this comment with the donation destination he selects, with further lamentations from me in due course.

Comment author: Gregory_Lewis 22 November 2017 08:10:02PM 6 points [-]

Carl has gotten back to me with where he would like to donate his gains, ill-gotten through picking on epistemic inferiors - akin to crocodiles in the Serengeti river picking off particularly frail or inept wildebeest on their crossing. The $1000 will go to MIRI.

With cognitive function mildly superior to the median geriatric wildebeest, I can take some solace that these circumstances imply this sum is better donated by him than I, and that MIRI is doing better on a crucial problem for the far future than I had supposed.

Comment author: RyanCarey 09 November 2017 11:39:01AM 5 points [-]