Comment author: Brian_Tomasik 14 January 2017 05:45:08PM 3 points [-]

I'm surprised to hear that people see criticizing EA as incurring social costs. My impression was that many past criticisms of EA have been met with significant praise (e.g., Ben Kuhn's). One approach for dealing with this could be to provide a forum for anonymous posts + comments.

Comment author: jsteinhardt 14 January 2017 07:24:21PM 4 points [-]

In my post, I said

anything I write that wouldn't incur unacceptably high social costs would have to be a highly watered-down version of the original point, and/or involve so much of my time to write carefully that it wouldn't be worthwhile.

I would expect that conditioned on spending a large amount of time to write the criticism carefully, it would be met with significant praise. (This is backed up at least in upvotes by past examples of my own writing, e.g. Another Critique of Effective Altruism, The Power of Noise, and A Fervent Defense of Frequentist Statistics.)

Comment author: Ben_Todd 12 January 2017 09:17:08PM 8 points [-]

though I do think that there are other groups / organizations that substantially outperform EA, which provides an existence proof that one can do much better

Interesting. Which groups could we learn the most from?

Comment author: jsteinhardt 13 January 2017 08:30:50AM 4 points [-]

I think parts of academia do this well (although other parts do it poorly, and I think it's been getting worse over time). In particular, if you present ideas at a seminar, essentially arbitrarily harsh criticism is fair game. Of course, this is different from the public internet, but it's still a group of people, many of whom do not know each other personally, where pretty strong criticism is the norm.

My impression is that criticism has traditionally been a strong part of Jewish culture, but I'm not culturally Jewish so can't speak directly.

I heard that Bridgewater did a bunch of stuff related to feedback/criticism but again don't know a ton about it.

Of course, none of these examples address the fact that much of the criticism of EA happens over the internet, but I do feel that some of the barriers to criticism online also carry over in person (though others don't).

Comment author: jsteinhardt 12 January 2017 07:19:44PM 19 points [-]

I strongly agree with the points Ben Hoffman has been making (mostly in the other threads) about the epistemic problems caused by holding criticism to a higher standard than praise. I also think that we should be fairly mindful that providing public criticism can have a high social cost to the person making the criticism, even though they are providing a public service.

There are definitely ways that Sarah could have improved her post. But that is basically always going to be true of any blog post unless one spends 20+ hours writing it.

I personally have a number of criticisms of EA (despite overall being a strong proponent of the movement) that I am fairly unlikely to share publicly, due to the following dynamic: anything I write that wouldn't incur unacceptably high social costs would have to be a highly watered-down version of the original point, and/or involve so much of my time to write carefully that it wouldn't be worthwhile.

While I'm sympathetic to the fact that there's also a lot of low-quality / lazy criticism of EA, I don't think responses that involve setting a high bar for high-quality criticism are the right way to go.

(Note that I don't think that EA is worse than is typical in terms of accepting criticism, though I do think that there are other groups / organizations that substantially outperform EA, which provides an existence proof that one can do much better.)

9

Individual Project Fund: Further Details

[Cross-posted from my blog .] In my post on where I plan to donate in 2016 , I said that I would set aside $2000 for funding promising projects that I come across in the next year: The idea behind the project fund is … [to] give in a low-friction... Read More
Comment author: Peter_Hurford  (EA Profile) 29 December 2016 07:14:04PM 2 points [-]

Curious to hear more about why you're using the donor lottery - that seems to be the only part you did not explain.

Also, while I did not expect it to be the case going in, I found your explanation for splitting your donation to be compelling.

Comment author: jsteinhardt 30 December 2016 08:46:07PM 2 points [-]

Thanks. I think my reasons are basically the same as those in this post: http://effective-altruism.com/ea/14d/donor_lotteries_demonstration_and_faq/.

Comment author: benmusch 28 December 2016 03:38:42PM *  1 point [-]

Want to add this to #1:

a) Death & sickness are bad for the economy. It's pretty uncontested that more people use malaria nets than when they are provided for free (can link research on this if necessary). When someone is sick, it's time that they can't spend in the labor force. When someone gets sick a lot as a child, it affects them so that they are a less productive worker in the future. When your child is sick, you have to spend time taking care of them that could have otherwise been hours you earned a wage, spent on goods, etc. So in that regard, this is probably an outweighing factor to a couple jobs.

b) Even if malaria nets don't effect wages and productivity at all, the money that would have gone to the malaria net maker doesn't just disappear, it's simply spent somewhere else. So jobs don't go away, they are just created in other areas.

So it's not just weighing job creation vs. malaria prevention, it's that malaria prevention probably helps job creation.

Comment author: jsteinhardt 28 December 2016 05:07:03PM 1 point [-]

So jobs don't go away, they are just created in other areas.

This isn't really true. Yes, probably there is some job replacement so that the jobs don't literally disappear 1-for-1. But there will probably be fewer jobs, and I don't think it's easy to say (without doing some research) whether it's 0.1 or 0.5 or 0.9 fewer jobs for each malaria net maker that goes away.

14

My Donations for 2016

[Cross-posted from by blog .] The following explains where I plan to donate in 2016, with some of my thinking behind it. This year, I had $10,000 to allocate (the sum of my giving from 2015 and 2016, which I lumped together for tax reasons; although I think this was... Read More
Comment author: jsteinhardt 11 December 2016 10:46:41AM *  2 points [-]

I like this idea. One danger (in both directions) with comparing to VC is that my impression is venture capital is way more focused on prestige and connections than funding charities is. In particular, if you can successfully become a prestigious, well-connected VC firm, then all of the Stanford/MIT students (for instance) will want you to fund their start-up, and picking with only minimal due diligence from among that group is likely to already be fairly profitable. [Disclaimer: I'm only tangentially connected to the VC world so this could be completely wrong, feel free to correct me.]

If this is true, what should we expect to see? We should expect that (1) VCs put in less research than OpenPhil (or similar organizations) when making investments, (2) hits-based is very successful for VC firms conditioned on having a strong established reputation. I would guess that both of these are true, though I'm unsure of the implications.

Comment author: jsteinhardt 02 December 2016 04:39:08AM *  3 points [-]

I feel like I care a lot about theory-building, and at least some of the other internal and external reviewers care a lot about it as well. As an example, consider External Review #1 of Paper #3 (particularly the section starting "How significant do you feel these results are for that?"). Here are some snippets (link to document here):

The first paragraph suggests that this problem is motivated by the concern of assigning probabilities to computations. This can be viewed as an instance of the more general problems of (a) modeling a resource-bounded decision maker computing probabilities and (b) finding techniques to help a resource-bounded decision maker compute probabilities. I find both of these problems very interesting. But I think that the model here is not that useful for either of these problems. Here are some reasons why:

It’s not clear why the properties of uniform coherence are the “right” ones to focus on. Uniform coherence does imply that, for any fixed formula, the probability converges to some number, which is certainly a requirement that we would want. This is implied by the second property of uniform coherence. But that property considers not just constant sequences of formulas, but sequence where the nth formula implies the (n+1)st. Why do we care about such sequences? [...]

The issue of computational complexity is not discussed in the paper, but it is clearly highly relevant. [...]

Several more points are raised, followed by (emphasis mine):

I see no obvious modification of uniformly coherent schemes that would address these concerns. Even worse, despite the initial motivation, the authors do not seem to be thinking about these motivational issues.

For another example, see External Review #1 of Paper #4 (I'm avoiding commenting on internal reviews because I want to be sensitive to breaking anonymity).

On the website, it is promised that this paper makes a step towards figuring out how to come up with “logically non-omniscient reasoners”. [...]

This surely sounds impressive, but there is the question whether this is a correct interpretation of Theorem 5. In particular, one could imagine two cases: a) we are predicting a single type of computation, and b) we are predicting several types of computations. In case (a), why would the delays matter in asymptotic convergence in the first place? [...] In case (b), the setting that is studied is not a good abstraction: in this case there should be some “contextual information” available to the learner, otherwise the only way to distinguish between two types of computations will be based on temporal relation, which is a very limiting assumption here.

To end with some thoughts of my own: in general, when theory-building I think it is very important to consider both the relevance of the theoretical definitions to the original problem of interest, and the richness of what can actually be said. I don't think that definitions can be assessed independently of the theory that can be built from them. At the danger of self-promotion, I think that my own work here, which makes both definitional and theoretical contributions relevant to ML + security, does a good job of putting forth definitions and justifying them (by showing that we can get unexpectedly strong results in the setting considered, via a nice and fairly general algorithm, and that these results have unexpected and important implications for initially unrelated-seeming problems). I also claim that this work is relevant to AI safety but perhaps others will disagree.

Comment author: jsteinhardt 02 December 2016 05:54:02AM 2 points [-]

Also, I realized it might not be clear why I thought the quotes above are relevant to whether the reviews addressed the "theory-building" aspect. The point is it seems to me that the quoted parts of the reviews are directly engaging with whether the definitions make sense / the results are meaningful, which is a question about the adequacy of the theory for addressing the claimed questions, and not of its technical impressiveness. (I could imagine you don't feel this addresses what you meant by theory-building, but in that case you'll have to be more specific for me to understand what you have in mind.)

Comment author: Owen_Cotton-Barratt 01 December 2016 12:07:21PM 6 points [-]

Think this is at least partially my fault. I included a phrase "(in the metric of papers written, say)" when discussing progress in the above post, but I didn't really think this was the main metric you were judging things on. I'll edit that out.

The sense it which it felt "bemusingly unfair" was that the natural situation it brought to mind was taking a bright grad student, telling them to work on AI safety and giving them no more supervision, then waiting 1-3 years. In that scenario I'd be ecstatic to see something like what MIRI have done.

I don't actually think that's the claim that was intended either, though. I think the write-up was trying to measure something like the technical impressiveness of the theorems proved (of course I'm simplifying a bit). There is at least something reasonable in assessing this, in that it is common in academia, and I think is often a decent proxy for How good are the people doing this work?, particularly if they're optimising for that metric. In doing so it also provided some useful information to me, because I hadn't seriously tried to assess this.

However, it isn't the metric I actually care about. I'm interested in their theory-building rather than their theorem-proving. I wouldn't say I'm extremely impressed by them on that metric, but at least enough that when I interpreted the claim as being about theory-building, I felt it was quite unfair.

Very interested to know whether you think this is a fair perspective on what was actually being assessed.

Comment author: jsteinhardt 02 December 2016 04:39:08AM *  3 points [-]

I feel like I care a lot about theory-building, and at least some of the other internal and external reviewers care a lot about it as well. As an example, consider External Review #1 of Paper #3 (particularly the section starting "How significant do you feel these results are for that?"). Here are some snippets (link to document here):

The first paragraph suggests that this problem is motivated by the concern of assigning probabilities to computations. This can be viewed as an instance of the more general problems of (a) modeling a resource-bounded decision maker computing probabilities and (b) finding techniques to help a resource-bounded decision maker compute probabilities. I find both of these problems very interesting. But I think that the model here is not that useful for either of these problems. Here are some reasons why:

It’s not clear why the properties of uniform coherence are the “right” ones to focus on. Uniform coherence does imply that, for any fixed formula, the probability converges to some number, which is certainly a requirement that we would want. This is implied by the second property of uniform coherence. But that property considers not just constant sequences of formulas, but sequence where the nth formula implies the (n+1)st. Why do we care about such sequences? [...]

The issue of computational complexity is not discussed in the paper, but it is clearly highly relevant. [...]

Several more points are raised, followed by (emphasis mine):

I see no obvious modification of uniformly coherent schemes that would address these concerns. Even worse, despite the initial motivation, the authors do not seem to be thinking about these motivational issues.

For another example, see External Review #1 of Paper #4 (I'm avoiding commenting on internal reviews because I want to be sensitive to breaking anonymity).

On the website, it is promised that this paper makes a step towards figuring out how to come up with “logically non-omniscient reasoners”. [...]

This surely sounds impressive, but there is the question whether this is a correct interpretation of Theorem 5. In particular, one could imagine two cases: a) we are predicting a single type of computation, and b) we are predicting several types of computations. In case (a), why would the delays matter in asymptotic convergence in the first place? [...] In case (b), the setting that is studied is not a good abstraction: in this case there should be some “contextual information” available to the learner, otherwise the only way to distinguish between two types of computations will be based on temporal relation, which is a very limiting assumption here.

To end with some thoughts of my own: in general, when theory-building I think it is very important to consider both the relevance of the theoretical definitions to the original problem of interest, and the richness of what can actually be said. I don't think that definitions can be assessed independently of the theory that can be built from them. At the danger of self-promotion, I think that my own work here, which makes both definitional and theoretical contributions relevant to ML + security, does a good job of putting forth definitions and justifying them (by showing that we can get unexpectedly strong results in the setting considered, via a nice and fairly general algorithm, and that these results have unexpected and important implications for initially unrelated-seeming problems). I also claim that this work is relevant to AI safety but perhaps others will disagree.

View more: Next