This week I'm at CVPR — the IEEE's Computer Vision and Pattern Recognition Conference, which is a huge AI event. I'm currently rehearsing the timing of my talk one last time, but I wanted to take a minute between run-throughs to link to my co-author Steven Forsyth's wonderful post on the NVIDIA research blog about our paper.
Steven does a fantastic job of describing our work, so head over there to see what he has to say. I couldn't resist putting a post of my own because (a) I love this video we created...
...and (b), Steven left out what I think was the most convincing result we had, which shows that BaRT achieves a Top-1 accuracy on ImageNet that is higher than the Top-5 accuracy of the previous state-of-the-art defense, Adversarial Training.
Accuracy of BaRT under attack by PGD for varying adversarial distances, compared to the previous state-of-the-art.
Also, (c) I am very proud of this work. It's been an idea I've been batting around for almost three years now, and I finally got approval from my client to pursue it last year. It turns out it works exactly how I expected, and I can honestly say that this is the first — and probably only — time in my scientific career that has ever happened.
If you want a copy of the paper, complete with some code in the appendices, ((Our hands are somewhat tied releasing the full code due to the nature of our client relationship with the wonderful Laboratory for Physical Sciences, who funded this work.)) our poster, and the slides for our oral presentation you can find it on the BaRT page I slapped together on my website.
I would think we've saturated the "modern re-tellings of fairytales, but for adults" genre, but this was supremely good. They reminded me of Garrison Keillor in the way that some sadness or loss was mixed in to the stories without them being outright tragic.
(I've had this post sitting in my drafts for a very long time. How long? Since well before we all found out Keillor was a creep. So... I guess I'll amend the above to "it reminds me pre-2017 Garrison Keillor"? It's been about 15 years since I read any of his stories, so maybe I should just scrap this reference all together? Screw it.)
The View from the Cheap Seats, Neil Gaiman
A collection of non-fiction pieces: essays, transcripts of awards speeches, introductions, forwards, etc. Some felt dated, but most I can safely call "timeless." Many of them did make me want to go read the various books or authors that he was commenting on (e.g. Jeff Smith, Samuel R Delaney, Fritz Leiber, Dunsany) which seems like as good a thing as can be said about an introduction to a book. The final piece is a memorial to his friend and collaborator, Terry Pratchett, titled "A Slip of the Keyboard." It is definitely worth reading especially for Pratchett fans.
The Liberation, Ian Tregillis
This is the conclusion to Tregillis' "Mechanicals" trilogy. I found the whole series good, but not nearly as good as his "Bitter Seeds" series. "Bitter Seeds" had plot points and story lines that were woven complexly, foreshadowed with subtlety, and epic emotional highs and lows. The Mechanicals was good, but had little of that finesse.
"Mechanicals" is focused on free will and robots. It's an interesting concept, and a good way of using sci-fi to explore ideas. (Which, I suppose, is why it's been done plenty of times.) If I was a writer, I would like to do a similar story about robots, but instead of free will it would be about depression. Inside Out had one of the better depictions of depressions I've seen on screen. Depression — in my experience — isn't just regular sadness turned up to eleven. It's feeling nothing at all. Mechanical androids seem like a perfect vehicle to explore that. Instead of robots fighting to be able to act on their own preferences or desires or motivation, they would be fighting to be able to have preferences or desires or motivations in the first place.
The Gene: An Intimate History, Siddhartha Mukherjee
Also not as good as his previous work, The Emperor of All Maladies: A Biography of Cancer, but still very, very good. As in Emperor of All Maladies, Mukherjee does a great job of blending history, science, and his own personal experiences.
I did not appreciate before reading this exactly how quickly the concept of genetics has grown. The hundred years following Darwin's work in the 1850s and Mendel in the 1850s and 1860s was head-spinningly prolific. I had also not considered that eugenics was at the very forefront of applied genetics. I had thought of eugenics as a weird sideline (indeed, I wish it had been) but according to Mukherjee's telling it was at the very center of genetics from its infancy. ((Mukherjee also does good work in not letting us get away with thinking eugenics was something unique to the Nazis; Brits and Americans were leading members of the eugenics travesty. We should confront the ugly parts of our history, where "we" is both national groups as well as ideological ones like, in this case, progressives and High Rationalists.))
Mukherjee's discussion of penetrance (the way specific genes only affect people in probabilistic ways) was very good. I wish this concept was more widely appreciated, as compared to the binary "you have a mutation or you don't" level of understanding that is common.
Mukherjee also hammers home the idea that a mutation can not be judged to be good or bad by itself, but must be evaluated in the context of a given organism in a given environment. This is important for genetics, but important much more broadly. In my own work I've had to explain many times that certain behaviors of a neural network can not be judged in isolation. They can only be evaluated in the context of the data sets they're operating on and the tasks they're being asked to do.
I found Mukherjee to be on weakest footing when discussing the ethical implications. He seems to be engaging in too much mood affiliation.
Medieval Europe, Chris Wickham
I was looking for a good overview of medieval history. I've learned isolated pieces here and there, but my secondary education covered exactly zero European history, so I'm lacking a broad outline. This wasn't that really that book. It did a good job of describing major political themes but didn't mention any specific events. It's a valuable approach, but not the one I expected. The focus was mainly on state capacity of the different regions. (Which I actually think is a very valuable approach, just not what I was looking for.)
One take-away: France is very fortunate to have inherited Roman roads. That gave them a big leg-up in state capacity compared to their central and eastern rivals.
The Aeronaut's Windlass, Jim Butcher
This is the first in a new series in a victorian, pseudo-steampunk setting. Butcher is generally a fun read, and this is no exception. It's nice to see some fantasy novels that aren't in either a modern time period or a Tolkinesque medieval era.
I don't have a ton to say except that there were Aeronauts but there was no windlass. Is the title a metaphor that is going over my head, or is it just a catchy phrase without relation to the story?
Oh, also one thing in the world building got under my skin. Everyone in the story lives in these towers constructed by "the ancients" or some such, because the surface of the planet is poisonous and/or infested with ravenous hellbeasts. Each tower is a city-state, and people fly between them on airships. As a result, Butcher mentions over and over how much of a luxury resource wood is, because it's risky to go to the surface for timber. But what about all the other raw materials?? Where are they getting metal? Cotton? Wool? A huge library plays a role in the story; what are they making paper out of? Ships are described with complicated rigging; what is rope made from?He mentions that meat is vat-grown and therefor rare, but what about all the other food? Why is wood singled out as the one luxury?
Waking Gods, Sylvain Neuvel
This is the sequel to Neuvel's Sleeping Giants. Very good. Told in the same style, i.e. each chapter is a diary entry, interview transcript, communication intercept, news report, etc. which reveals the story to you little by little. Points for a good story, and double points for non-standard narrative form.
The Rise and Fall of D.O.D.O., Neal Stephenson & Nicole Galland
This had much of Stephenson's cleverness without his extremely lengthy didactic digressions. I'm not sure how much of the book was Stephenson and how much was Galland, but the combination worked very well. Recommended. I'm very much hoping there will be a sequel, but it's not clear. Parts of it relating to academia and the defense/IC sectors did not quite square with what I've observed, but it's a novel about magic and supercomputers and time travel and parallel universes, so I think I can let that slide.
The Princess Bride: S. Morgenstern's Classic Tale of True Love and High Adventure, William Goldman
I love the movie, and I'm glad I finally got around to reading the book. As everyone knows, the book is almost always better than the book. This may be an exception. Either way, they are very close in quality, perhaps because Goldman also wrote the screenplay. (He also wrote Butch Cassidy and the Sundance Kid, and I never would have guessed that both of those were written by the same person.) The only obvious parts left out of the movie were some longer character back stories, which were helpful but not necessary.
The conceit of the book is that Goldman is merely the translator/editor of a story written by the fictitious S. Morgenstern. Goldman never lets this illusion slip. The forward, introduction, introduction to the anniversary edition, epilogue, footnotes and asides: the whole time he sticks to the notion that he's merely editing an existing book. He even weaves in true stories from his life as a screenwriter to further blur the lines. I love unreliable narrators, but this is my first experience with an unreliable author.
The Blade Itself, Before They Are Hanged, and Last Right of Kings, Joe Abercrombie
I plowed through all of the "First Law" trilogy almost back-to-back-to-back. Definitely recommended.
Usually when an author has multiple point-of-view characters and rotates chapters between them there are some story lines that are exciting and I want to get back to, and some I have to wade through to get back to the good bits. Not so here, especially in Before They Are Hanged. I also appreciated that there was not an obvious quest or goal that everyone was seeking. It was somewhat difficult to tell what the challenge for the various characters actually was. It all comes together in the end in a very satisfying way, but it was nice not having the constant score-keeping in the back of my head about "are we closer or farther from the Ultimate Goal of destroying the mcguffin/overthrowing the tyrant/winning the throne/whatever?"
Palimpsest: A History of the Written Word, Matthew Battles
Low on factual density. Highly stylized writing. I do give it points because the final and longest chapter, titled "Logos ex Machina," considers computer programs as a type of writing. Anything that is willing to give 10 Print a place in the history of writing is okay with me. Overall, there are better books on the history of book and language.
Crucial Conversations, Kerry Patterson, et al.
I read this as part of a quasi-book club at work. Some of the people at dinner said that it was difficult practice having these crucial conversations (i.e. high stakes, emotionally laden). I suggested that there is one easy way to get lots of experience with these conversations under your belt: get married.
I'd put this in to the better class of management book, in that it's worth reading but still spins twenty pages of valuable advice up to several hundred pages of content. The world would be a more efficient place if business people were willing to spend money at Hudson Books on management pamphlets instead of books.
Olympos and Illium, Dan Simmons
Just as grand in scope and ambition as Simmons' Hyperion series, but utlimately not as good. It took well into the second book for the pieces to start to fit together, and as a result of remaining in the dark I had a hard time carrying about what was going to happen next.
Seven Days in the Art World, Sarah Thornton
This was written in 2007, and revolves a lot &emdash; by necessity &emdash; around the intersection of art and money. I would love to see what would have changed if there was a post-crash follow up from 2009.
One chapter was a studio visit to Haruki Murakami's studio. This was an odd choice, since as the book makes clear he's a singularly weird artist since he spends so much of his time running a sort of branding agency. That made for interesting but unrepresentative material. I'd read a whole book composed of Thornton visiting different studios.
The Sea Peoples, S. M. Stirling
This was a let down compared to the dozen or so volumes in the series prior. The series started out with a classic speculative fiction approach: change one thing about the world and see what happens. (Modern technology stops working; neo-feudalism rises from the ashes.) Then in later volumes more mysticism was introduced to explain why the change happened, and to give some narrative structure and reason why the Baddies were so Bad. (Chaotic gods are using them as puppets to take over the world in a proxy fight against their Good God rivals.) But this latest installment is four fifths weird mystical fever dreams (literally) mixed up with homages to the King in Yellow (again, literally). It's off the rails. I'll still read the next volume, because I like my junkfood books and I enthusiastically commit the sunk costs fallacy when it comes to finishing book series. But still: off the rails.
To Rule the Waves: How the British Navy Shaped the Modern World, Arthur Herman
This was a very fun history. There's plenty of fact, but Herman does a good job of writing the "action scenes" of various engagements, for lack of a better word. His style is a little too Great Man-ish for me, but nonetheless this was a good read. There's also a non-zero chance he's overselling how important his subject matter is, but I could day that about 90% of non-fiction writers, and 99% of non-fiction writers who write about rather more obscure topics.
I would read an entire book about common English idioms with nautical origins. For example, lowering the sails on a ship is "striking sail." Sailors, who were paid chronically late by the Royal Navy, would refuse to let their ships leave harbor until they were paid back wages. To disable the ships, they would strike sail. Now a mass refusal to work is a strike.
I used to have this on my bedroom wall when I was a kid. That is a fact I bet you are happy that you now know.
It's a credit to Herman that I was a little emotional by the time I got to the end of the book. The Royal Navy keeps winning and winning, often against the odds, survives WWII and comes out victorious, and then is just... dismantled. It's probably the correct strategic/economic move, but that sort of unforced abdication is somewhat sad.
Of course I did grow up with a reproduction WWII-era Royal Navy morale poster on my bedroom wall, because my friend Eli brought it back from London for me, so I might be subconsciously nostalgic for the Royal Navy in a way most Americans are not.
Artemis, Andy Weir
Good, but not as good as The Martian. ((I feel like a lot of my reviews are "good, but not as good as their last book" (e.g. my reviews of Tregillis & Mukherjee, supra). This is probably not a terribly fair way to assess authors, but... eh. That's one way I judge books, and I think I'm not alone.)) I give Weir a huge amount of credit for writing a book that grapples with why people would want to live in space in the first place. A space colony is not an economically reasonable thing to do, and I don't like it when people hand-wave that problem away.
From here down, I'm just going to list some of the books I read in the last quarter or so of 2017 that I thought were vaguely interesting. They aren't any worse than the ones above, I just don't have time to write them up and I'm sick of this post sitting in my drafts folder.
Battling the Gods: Atheism in the Ancient World, Tim Whitmarsh
Afterlife, Marcus Sakey
How to be a Stoic, Massimo Pigliucci
Potato: A History of the Propitious Esculent, John Reader
Golden Age and Other Stories, Naomi Novik
Within the Sanctuary of Wings, Marie Brennan
Alphabetical: How Every Letter Tells a Story, Michael Rosen
Besieged, Kevin Hearne
Assassin's Apprentice, Robin Hobb
Stoicism Today (Volume One), Patrick Ussher et al.
I don't usually write up my technical work here, mostly because I spend enough hours as is doing technical writing. But a co-author, Jon Barker, recently wrote a post on the NVIDIA Parallel For All blog about one of our papers on neural networks for detecting malware, so I thought I'd link to it here. (You can read the paper itself, "Malware Detection by Eating a Whole EXE" here.) Plus it was on the front page of Hacker News earlier this week, which is not something I thought would ever happen to my work.
Rather than rehashing everything in Jon's Parallel for All post about our work, I want to highlight some of the lessons we learned from doing this about ML/neural nets/deep learning.
As way of background, I'll lift a few paragraphs from Jon's introduction:
The paper introduces an artificial neural network trained to differentiate between benign and malicious Windows executable files with only the raw byte sequence of the executable as input. This approach has several practical advantages:
No hand-crafted features or knowledge of the compiler used are required. This means the trained model is generalizable and robust to natural variations in malware.
The computational complexity is linearly dependent on the sequence length (binary size), which means inference is fast and scalable to very large files.
Important sub-regions of the binary can be identified for forensic analysis.
This approach is also adaptable to new file formats, compilers and instruction set architectures—all we need is training data.
We also hope this paper demonstrates that malware detection from raw byte sequences has unique and challenging properties that make it a fruitful research area for the larger machine learning community.
One of the big issues we were confronting with our approach, MalConv, is that executables are often millions of bytes in length. That's orders of magnitude more time steps than most sequence processing networks deal with. Big data usually refers to lots and lots of small data points, but for us each individual sample was big. Saying this was a non-trivial problem is a serious understatement.
Architecture of the malware detection network. (Image copyright NVIDIA.)
Here are three lessons we learned, not about malware or cybersecurity, but about the process of building neural networks on such unusual data.
1. Deep learning != image processing
The large majority of the work in deep learning has been done in the image domain. Of the remainder, the large majority has been in either text or speech. Many of the lessons, best practices, rules of thumb, etc., that we think apply to deep learning may actually be specific to these domains.
For instance, the community has settled around narrow convolutional filters, stacked with a lot of depth as being generally the best way to go. And for images, narrow-and-deep absolutely seems to be the correct choice. But in order to get a network that processes two million time steps to fit in memory at all (on beefy 16GB cards no less) we were forced to go wide-and-shallow.
With images, a pixel values is always a pixel value. 0x20 in a grayscale image is always darkish gray, no matter what. In an executable, a byte values are ridiculously polysemous: 0x20 may be part of an instruction, a string, a bit array, a compressed or encrypted values, an address, etc. You can't interpolate between values at all, so you can't resize or crop the way you would with images to make your data set smaller or introduce data augmentation. Binaries also play havoc with locality, since you can re-arrange functions in any order, among other things. You can't rely on any Tobbler's Law ((Everything is related, but near things are more related than far things.)) relationship the way you can in images, text, or speech.
2. BatchNorm isn't pixie dust
Batch Normalization has this bippity-boppity-boo magic quality. Just sprinkle it on top of your network architecture, and things that didn't converge before now do, and things that did converge now converge faster. It's worked like that every time I've tried it — on images. When we tried it on binaries it actually had the opposite effect: networks that converged slowly now didn't at all, no matter what variety of architecture we tried. It's also had no effect at all on some other esoteric data sets that I've worked on.
We discuss this at more length in the paper (§5.3), but here's the relevant figure:
KDE plots of the convolution response (pre-ReLU) for multiple architectures. Red and orange: two layers of ResNet; green: Inception-v4; blue: our network; black dashed: a true Gaussian distribution for reference.
This is showing the pre-BN activations from MalConv (blue) and from ResNet (red & orange) and Inception-v4 (green). The purpose of BatchNorm is to output values in a standard normal, and it implicitly expects inputs that are relatively close to that. What we suspect is happening is that the input values from other networks aren't gaussian, but they're close-ish. ((I'd love to be able to quantify that closeness, but every test for normality I'm aware of doesn't apply when you have this many samples. If anyone knows of a more robust test please let me know.)) The input values for MalConv display huge asperity, and aren't even unimodal. If BatchNorm is being wonky for you, I'd suggest plotting the pre-BN activations and checking to see that they're relatively smooth and unimodal.
3. The Lump of Regularization Fallacy
If you're overfitting, you probably need more regularization. Simple advice, and easily executed. Everytime I see this brought up though, people treat regularization as if it's this monolithic thing. Implicitly, people are talking as if you have some pile of regularization, and if you need to fight overfitting then you just shovel more regularization on top. It doesn't matter what kind, just add more.
We ran in to overfitting problems and tried every method we could think of: weight decay, dropout, regional dropout, gradient noise, activation noise, and on and on. The only one that had any impact was DeCov, which penalized activities in the penultimate layer that are highly correlated with each other. I have no idea what will work on your data — especially if it's not images/speech/text — so try different types. Don't just treat regularization as a single knob that you crank up or down.
I hope some of these lessons are helpful to you if you're into cybersecurity, or pushing machine learning into new domains in general. We'll be presenting the paper this is all based on at the Artificial Intelligence for Cyber Security (AICS) workshop at AAAI in February, so if you're at AAAI then stop by and talk.
All programming blogs need at least one post unofficially titled “Indisputable Proof That I Am Awesome.” These are usually my favorite kind of read, as the protagonist starts out with a head full of hubris, becomes mired in self-doubt, struggles on when others would have quit, and then ultimately triumphs over evil (that is to say, slow or buggy computer code), often at the expense of personal hygiene and/or sanity.
I'm a fan of the debugging narrative, and this is a fine example of the genre. I've been wrestling with code for mapping projections recently, so I feel Miller's pain specifically. In my opinion the Winkel Tripel is mathematically gross, but aesthetically unsurpassed. Hopefully I'll find some time in the next week or so to put up a post about my mapping project.
A screenshot of a project I've been working on to map geotagged tweets.
File under "all college degrees are not created equal" or perhaps "no, junior, you may not borrow enough to buy a decent house in order to get a BA in psych."
Social cohesion can be thought of as a manifestation of how "iterated" people feel their interactions are, how likely they are to interact with the same people again and again and have to deal with long term consequences of locally optimal choices, or whether they feel they can "opt out" of consequences of interacting with some set of people in a poor way.
Munger links to some very good analysis but it occurs to me that what is really needed is the variance of grades over time and not just the mean. (Obviously these two things are related since the distribution is bounded by [0, 4]. A mean which has gone from 2.25 to 3.44 will almost certainly result in less variance here.)
I don't much care where the distribution is centered. I care how wide the distribution is — that's what lets observers distinguish one student from another. Rankings need inequality. Without it they convey no information.
I share Graur's and Tabarrok's wariness over "high impact false positives" in science. This is a big problem with no clear solutions.
The Graur et al. paper that Tabarrok discusses is entertaining in its incivility. Sometimes civility is not the correct response to falsehoods. It's refreshing to see scientists being so brutally honest with their opinions. Some might say they are too brutal, but at least they've got the honest part.
McCaffrey is completely right. But good luck to him reasoning people out of an opinion they were never reasoned into in the first place.
I do like the neologism "sustainable pricing" that he introduces. Bravo for that.
I would add a sixth reason to his list: accusations of "price gouging" are one rhetorical prong in an inescapable triple bind. A seller has three MECE choices: price goods higher than is common, the same as is common, or lower than is common. These choices will result in accusations of price gouging, collusion, and anti-competitive pricing, respectively. Since there is no way to win when dealing with people who level accusations of gouging, the only sensible thing to do is ignore them.
Executive summary: apiarists have agency, and the world isn't static. If the death rate of colonies increases, they respond by creating more colonies. Crisis averted.
"Betting Therapy" should be a thing. You go to a betting therapist and describe your fears — everything you're afraid will happen if you do X — and then the therapist offers to bet money on whether it actually happens to you or not. After you lose enough money, you stop being afraid.
