Research Scientist. If you wan to do a more research elated role, I'd suggest you find one. They're certainly out there, just talk to companies who are hiring (see the NeurIPS sponsor's page). I can't tell you how hard/easy it will be to get in if you don't have any recent, relevant publications in the area, though. I'd expect that at least early-stage startups might not care, but you won't have time to do research there, either.

I've seen people have a lot of success by doing a PhD in their respective field (e.g. Chemistry, Biology, Hydrology, ....) that applies ML to their field, and then slowly change their research to ML (e.g. going from "predicting stuff about atoms" to "using super state-of-the-art ML to predict stuff" to "adapting state-of-the-art stuff in ways that might be useful to other ML people and publishing at NeurIPS". You'll need to discuss this with your advisor though, and ideally find one who's willing to support you (or at the very least doesn't mind if you try).

I haven't checked

Look for someone whose current students seem happy (talk to the students if you can!). The ideal advisor is someone who values work/life balance, yet still manages to do good work, and is willing to talk to you/help you on a regular basis.

I might be misremembering, but didn't Sperbank have some people at their booth whose job seemed to consist purely of being eye candy?

The poster sessions are still nice, though overcrowded. I didn't particularly enjoy the keynotes so far. But in general terms, it's still fairly academic.

How is that operationalizing it?

new number who 'dis?

Underrated: I haven't read all of the Outstanding Papers yet, but I'm looking forward to digging into "Is Out-of-distribution Detection Learnable?" and "A Neural Corpus Indexer for Document Retrieval". (though arguably oustanding papers aren't underrated ;) ).

Cohere: Aidan wasn't there, which was a bit said, would've enjoyed meeting him again.

On the plus-side, the Mug looks nicer than the 2019 ones. On the negative side, a lot of the usual faces are missing. E.g. haven't seen Yoshua Bengio, or Neil Lawrence, or LeCun, or Schmidhuber, or Schoelkopf, Ilya Sutskever or Ian Goodfellow or lots of other folk. Not that there aren't a ton of brilliant people regardless, but it's kind of weird (might have just missed some of them in the crowd, though).

Poster sessions are crowded like 2019, but it empties out fairly quickly (e.g. 1 hour after the start of the session there's noticable less people). No-one's social distancing and <30% of people wore masks.

No Sperbank models this year. Nvidia is also missing, which was a bit more unexpected.

They are for everyone who gets an invite. How selective those are and who gets one depends on the company. The general rule is that the more hireable you are, the more they'll be interested in inviting you. The best way to know about them is to talk to the people at the expo booths. Some are very public, e.g. Google let's everyone in who scans the QR code at their booth in time. On the other hand at GResearch you had to sweet talk the recruiter. I stopped talking to the booth people long ago, so I miss out on most events.

So, maybe let's not put the cart in front of the horse, focus on getting through the PhD first. You sound super motivated, but just to be sure, these are your two most important next steps:

1. If you can, find a good advisor/mentor. They'll introduce you to people and help you develop your research skills. TBH i don't know if the current PhD student job market is more of a "take what you can get" or a "lots of open positions" thing. But if you have several options on where to join, go with the ones that you feel like you can vibe with: they should have experience with publishing on top tier conferences, but also willing to help you succeed. use the search function, lots of advice on this reddit for what makes a good advisor.

2. Decide on what kind of research you want to do/pick a topic. Would you rather do applied research or theoretical stuff? There's a whole spectrum from maths or learning theory to foundational (e.g. RL or deep learning) to applied (computer vision, NLP, ...). Find a topic that excites you enough that you can dedicate several years to learning it and excelling in a subfield of it. The classes you'll be taking and the conversations with people at your lab/conferences should give you a flavor of what's out there. There's a bit of luck involved with picking a research direction that proves to be relevant, so advise from advisors/mentors helps a ton at that particual stage. One of the most important lesions you should learn in your PhD is how to find and approach good research questions.

Those are the two most important things to optimize your PhD success, which in turn optimizes hireability. In general your best bet at an industry research position is to do work that is meaningful enough that someone notices, in a field that the company (or a reasearch team with headcount) cares about. What that is depends on where you want to go and which field you want to work in. Definitely try to collaborate with industry, apply for internships or similar programs, or try to find other ways to collaborate while you're still in your phd. But like I said: I'd focus more on enjoying my PhD first, everything else should merely be a regularizer. Find a topic that interests you and the rest will follow.

So obvious joke first: no, I don't agree because that's a continuous Random Variable and you're asking for a point estimate. badum tss

No but seriously, no-one can remotely predict scientific advances 10 years into the future... I don't have a good notion of what consciousness for an AI would look like. The definition Chalmers gave today ("experiencing subjective awareness") is a bit too wishy-washy, how do you measure that? But broadly speaking I don't think we'll have self-aware programs in 10 years.

Only the leap between the beginning of the conference hall, and the beginning of NeurIPS at the end of that hall. All the big papers so far have been known for long enough.

Most haven't happened yet, so I don't know. GResearch is throwing one at a boat this year, which for now sounds like the most exciting thing this conference as far as Parties go. I.e., FlowRider's not gonna show up again, those times are over, now it's mostly about trying to get people to mingle. I hope the PhD students still party it up amongst themselves, but if so, I wasn't invited.

Apart from that, the biggest news in the ML research social sphere is that the 42. annual Workboat Show will begin tomorrow on the other side of the convention center. Everybody saw their big posters and everyone is talking about them. It's this year's inside joke. If no-one acknowledges them in keynotes or the end-of-conference speech, I'll be heavily disappointed.

Well, I don't know at what level you're at, but I'm assuming you're undergrad and will keep this high level:

Well, we have models that have some understanding of text (e.g. GPT-3), and some notion of images (anything since ResNet or even AlexNet). Mostly in the vague sense that when we feed text or images into these "encoder" networks, they spit out a "representation vector" (i.e., a bunch of hard to decrypt numbers). We can feed those into "decoder" networks that do sensible things with those vectors (e.g. tell you that this vector is most likely of class "husky" at this and that position, or produce text that is the logical continuation of whatever textprompt you give it). We can train huuuuuuuuuuuuuge models like that (Billions of parameters to learn, probably cost 10^6$ to train for the first time). Very recently (last 1-2 years) we've learned to combine these two models (e.g. CLIP). So you can feed in text and get out an image (e.g. Stable Diffusion), or feed in text and image and get out whatever the text said to get out of the image (e.g. Flamingo).

That's roughly where we are in terms of big picture. Currently, we're working on better ways to train these models (e.g. by requiring less to no supervised data), or find out how they scale with input data and compute, or get smaller models out of the big ones, or whether to name the big ones "foundational" or "pretrained", or find creative ways to use or improve e.g. Stable Diffusion and similar models for other applications like reading code, as well as bunch of other stuff. No idea what the next big idea will be after. My hunch is on memory (or rediscovering recurrent nets).

Edit: this was extremely Deep Learning centric, sorry. There's of course other stuff going on: I don't follow Reinforcement Learning (learning over time with rewards) at all, so no clue about that, though it's arguably important for getting from ML to more general AI. Also, there are currently lots of issues being raised wrt. Fairness and Biases in AI (though I have seen almost no papers on it this year, why is that?). And more and more, people start to reason about "causality", i.e., how to go from correlation of things to causation between things... lots of other stuff outside my bubble.

I totally understand your position. My hat is off to anyone who chooses to fight for Ukraine. I'd expect the Ukrainian military and nato allies to work on projects just like this, because it makes sense to have such a system. People who want to work on these projects full time should in fact just join the military to avoid duplicate work (and to get closer to the actual customers of your work).

I'm just saying that the work being done here is easier to abuse (and could potentially turn out to develop into a net negative long after this war is over) than work done by someone who eg. drives a tank. There are ethical considerations that might not be obvious at first sight.

The things they're looking to do can easily be used in offensive ways as well. Plus, we know from the Manhattan project that in hindsight many of the participants wished they hadn't done what they've done. Not that I think that this project is similarly large in scope, but the basic context of "don't work on warfare tech, even if it currently seems important" stands. I'm sure there are people disagreeing, otherwise no one would work in that sector or join the military. But I think there's a lot of people who are looking for ways to help Ukraine w/o realizing that work like this might have consequences beyond the current conflict. If you're building a weapon that your military uses to fight the enemy, then that has less possibilities for misuse (provided you trust your military) than if you develop a software to track humans in satellite images in real time and put it out there in the world for good and bad actors to use, alike.

(Also, "Your argument is childish, but I won't elaborate and you're a stoner" is middle school argument, too, hereComeThatGurl420)

I've said this once, but I'll happily say it again for anyone who thinks of participating: Fuck Putin with a dildo full of wooden splinters, but be aware that by joining this project you'll be working on warfare technology. And that whatever you'll develop now to help against Russia's invasion might later be used in other military conflicts, about which you might feel much more ethically ambiguous. The road to hell is paved with good intentions.

Knowing about subgradients (see other answers) is nice and all, but in the real world what matters is what your framework does. Last time I checked, both pytorch and jax say that the derivative of max(x, 0) is 0 when x=0.

I don't understand what I'm looking at.

1. At first glance, it seems like the Bundesliga has the least amount of variation (it's always Bayern Munich and only 2x BVB), while the Serie A has at least 3 different winners.

2. I asume think the title does not match the graph (and you're not really talking about the variance of who wins the title). Also, there seem to be some sort of error bars here, but they're also weird (e.g. for the Serie A they go upwards a lot but apparently not downwards, while it's the other way around for Ligue 1. Or does the blue line indicate something else?

3. what does PPG actually mean? Is this only for the team winning the league in that year? Or over all games? I'm confused.

So, in case you're not aware, matrix-matrix multiplication is THE workhorse of every BLAS implementation. I'm not too familiar with the Accelerate framework, but the really good implementations (e.g. MKL from Intel, or OpenBLAS) are extremely highly optimized (as in: there are people who are working on this professionally for years as their main job). You're very unlikely to get close to their performance, and shouldn't feel bad if they beat you by a lot.

I'd suggest giving OpenBLAS a whirl if you want to optimize for the absolute top achievable speeds. It's the best free BLAS implementation out there. For learning, googling for "cache optimized gemm" will give you good starting points on techniques for achieving SOTA performance in matrix-matrix multiplication.

so you rewrote all matrix products, without using BLAS?

EDIT: if so: why not use OpenBLAS instead (which afaik supports fp16 and bf16, too)?