Tag Archives: computer science

Book List: 2019Q2

I'll call this the Wizards & Cryptarchs, Frauds & Revolutions Edition —or— "What I've been reading when I'm not prepping for lectures and wrestling with toddlers."

Cover of Ellen Ullman's "Life in Code"

Life in Code, Ellen Ullman

When Ullman sticks to psychology, writing about what's like to be dealing with code, she is brilliant. No one communicates what it feels like to code as well as she does. When she expands the scope to sociology, she is mediocre. I can get that brand of techno-cynical, socialist dirigisme in any magazine on the newsstand. ((Here I'm not using "socialist" as the generic critique of any left-leaning idea that it is sometimes sloppily deployed as. Ullman is a former member of the Communist party; she has self-identified as a socialist.))

This is a collection of previously published essays. The good ones are great, the others are worth reading but not special. They were published over the last several decades, so merely seeing which themes and topics aren't addressed — social media is conspicuously absent from earlier discussion of the internet — is interesting in itself.

Meditation for Fidgety Skeptics, Dan Harris and Jeffrey Warren with Carlye Adler

I found this a little too squishy for my tastes, with its "everything can be meditation if you do it right" thesis. I'm saying this as someone who struggles in my meditation practice, so I completely get the strategy here. They want to provide a gentle on-ramp to get non-meditators to give it a shot. I've got nothing against that as a goal or a strategy, but I'd still like a little more discipline from meditation instruction. Meditation isn't supposed to be (too) easy. (What worth doing is?)

Nevertheless, this was pretty good. I've recommended it to my wife, because I think she's a good target audience, much like Dan Harris' own wife. I think she'll appreciate the way different chapters cover various objections to starting a meditation practice (I can't find the time; it's self-indulgent; etc.).

I especially liked the way the authors framed how you should respond when you notice you're "doing it wrong" while sitting: not "shame on me; you're doing a bad job meditating" but "good job for having the self-awareness to notice you're getting distracted."

Cover of "The Dark Forest"

The Dark Forest, Cixin Liu

This is the sequel to Three Body Problem. The first 40% or so of this volume was very slow, but the remainder picked up speed. My chief complaint is that there were no new ideas introduced in the first portion. The translator has changed between the first volume and this one, which may have also contributed to the torpid pacing. (I read TBP some months back, so I can't remember the specifics of what might be different in the translation.)

I'm not sure exactly how to say this, but Dark Forest struck me as being very "Chinese" — the focus on ideological purity and morale in the population, the interest in political affiliations amongst industry and the military, etc. It was interesting to read for that perspective alone.

Dark Forest revolves around what I think of as a Hari Seldon-esque view of social science that I always find off-putting. I'm tempted to say this is another element that is a result of Liu's Chinese heritage, but it crops up so often in SciFi that I can't do that. (I think the actual cause is that many SciFi authors who like physics want history & sociology to be as rigorous and reductionist as physics is.)

A Little History of Philosophy, Nigel Warburton

Every chapter of this brief book is about one important philosopher from history. Warburton does a good job of tying them all together into a single thread. It's nothing ground breaking, but makes for a pretty good introduction/refresher. I enjoyed listening to it on dog walks/commutes, because the structure was very digestible: I could listen to one chapter (ten or twelve minutes?), do some learning, and then move on to a different book on a different topic. I don't think I would have been in the mood currently to plow through four hours of Intro Philosophy lessons if I couldn't chunk it up like that.

I first heard of Warburton as the host of the Philosophy Bites podcast, in which he interviews other philosophy professors about their work. I haven't listened in a long time, but I still recommend it. The serious-but-approachable style he uses in that podcast carries through to this book.

No-Drama Discipline: The Whole-Brain Way to Calm the Chaos and Nurture Your Child's Developing Mind, Daniel J. Siegel & Tina Payne Bryson

I had the same reaction to this that I have to most of the business books I've read: there's a kernel of good information here, but the authors spend twice as long convincing me of how relevant and useful the advice is as they do just giving me the advice. I think they also play a little fast-and-loose with neuroscience for my taste. I didn't notice anything egregiously wrong — and I think they do know what they're talking about — but it felt like they were trying a little too hard to layer on scientific respectability by giving folksy descriptions of brain science.

Nevertheless, there's some pretty good advice here. I think it's difficult to write a book about discipling all kids between the ages of 2 and 18. Maybe my opinion on that will change once mine get out of the toddler stage.

I'm a Stranger Here Myself: Notes on Returning to America After Twenty Years Away, Bill Bryson

This is a collection of newspaper columns that Bryson wrote for an English paper after he moved back to America in the 90s. It's about what you would expect from a newspaper humorist. It's never as good as Bryson can really be, but it's pleasing in a homey, comforting way.

It was also nice to have an audiobook I could listen to where each chapter was only several hundred words. I could listen to one of these while knocking out a chore when a more complicated piece of narrative or non-fiction would be difficult to digest in four or five minute chunks.

The comparison of dealing with the Social Security Administration compared to the British immigration authority was the highlight for me.

Grave Peril, Jim Butcher

This is the fourth of Butcher's "Harry Dresden" novels. If you liked the first three, you'll probably like this. You'll probably like it more actually; Butcher seems to be getting better as a writer at this point in his publication history. Honestly I wasn't really paying enough attention to this book to be able to put my finger on why or how.

The Quantum Thief, Hannu Rajaniemi

This is the first in a trilogy about posthuman gentleman-thief Jean le Flambeur. I first read it years back (October 2014), and decided to re-read it before picking up volumes two and three since it was dense with weird concepts and novel vocabulary and allusions. (Cryptarchs! Exomemory! The Engineer-of-Souls! The Dilemma Prison! The All-Defector!) Even the second time through, I found it difficult to keep up with what was happening, since Rajaniemi gives little to no exposition to introduce you to all of the wonderful concepts and terms. Still, this is recommended for its creativity.

A Wizard of Earthsea, Ursula K. Le Guin

Cover to "Wizard of Earthsea"

I first read this when I was probably 10, and again in college, and now again when I'm 35. It not only holds up, I think I like it more.

It's extremely refreshing to read some fantasy that doesn't feel the need to bulk up to 800 pages with descriptions of the dishes at every feast and the heraldry of every noble family.

This is one of the books that I'm really looking forward to reading as a bed time story when the wee ones are old enough for "real" stories. ((By "real" I mean "not finished in a single sitting."))

I can not recommend this enough if you haven't read it before.

Here's some of the Daoist-flavored aphorisms from Wizard of Earthsea that I wrote down this time around:

  • Manhood is patience. Mastery is nine times patience.
  • The wise needn't ask, the fool asks in vain.
  • For a word to be spoken, there must be silence. Before, and after.
  • To light a candle is to cast a shadow.

I've been leaning on that first one really hard with the two previously-mentioned toddlers in the house right now.

Pushing Ice, Alastair Reynolds

This is the second Reynolds books I've read in as many months, and I am a full-on convert now. So much fun.

The preface is set several tens of millennia in the future, and seems to give away what is going to go wrong for our protagonists in the present. The adventure comes from figuring how exactly things go wrong and how they deal with it. Then as you reach about the mid-point of the book or slightly after you realize that that disaster was only the first part of the challenge and things get much weirder than you imagined.

Like House of Suns, Reynolds captures the immensity of space and its psychological impact in a way that few other scifi authors have.

My only complaint is that most of the story takes place in a society of a couple of hundred people, but it seems to have the economic and social structure of a much, much larger population. Would a couple of Dunbars worth of people really be able to support restaurant entrepreneurship, as Reynolds mentions off-hand?

Cover of "Bad Blood" by John Carreyrou

Bad Blood, John Carreyrou

I devoured this over the course of a couple of plane rides. It has gotten tons of praise, all of it deserved. I'm not sure what to add.

I have no earthly idea how so many investors and partners would not demand to see a working demo of Theranos' capabilities. I really cannot conceive of agreeing to back a system without being able to compare it to a baseline. "Sounds great. Here's two blood draws from twenty people. You take half of each pair and I'll send the other to a conventional lab. I'm so excited to see how well they match up!" Is that so hard? I don't think I would even take a second meeting — to say nothing of investing nine figures — without seeing the results from that.

Maybe I'm an outlier since so much of what I do day-to-day is about replication? Maybe this is hindsight bias on my part? Maybe I would be just as hoodwinked by Elizabeth Holmes' reported charisma?

A lot of Theranos' success is keeping the scam going was because so many of the insiders who saw through things were bullied into remaining quiet. Many of those who did want to blow the whistle were unable to exfil evidence. Would society be better or worse off if more people had the tradecraft to get the relevant documents out of places like Theranos? I'm guessing this would be a net negative since it would also allow more industrial espionage, more insider trading, etc., but the people who are motivated to do such things also have the motivation to learn how to get away with them, whereas it's the innocent potential whistleblowers who have never thought of a more advanced way to smuggle/maintain data besides "forward it to my personal email account" who are left without tools.

The Pursuit of Italy: A History of a Land, Its Regions, and Their Peoples, David Gilmour

Cover of "The Pursuit of Italy" by David Gilmour

This was a history of Italy-as-a-concept rather than Italy-as-a-geographic-region or Italy-as-a-nation-state. As Massimo d'Azeglio said after the Risorgimento: "We have made Italy. Now we must make Italians." The existence of a single country called "Italy" is highly historically contingent, and The Pursuit of Italy explores that contingency.

I had thought this was primarily going to be about the unification process of the Risorgimento, since I had heard it quoted so heavily in a podcast on that topic. ((Talking History: The Italian Unification.)) Gilmour actually covers much more ground, both before and afterwards. The book continues all the way up to discussions of contemporary politics, including Berlusconi, the Northern League, etc. I think I actually found the period preceding and following unification to be more interesting, as those chapters were more about culture than politics.

One take-away is that almost every Italian political leader since unification seems like a fool, including those with massive public monuments scattered across the country. (Exceptions: Garibaldi seems like a tactical dunce but strategically sharp and extremely charismatic, as well as principled. Mazzini also seems intelligent and principled, but he spent most of his life in exile as persona non grata and has not been fully retconned into the pantheon of Italian founding fathers.) Other than those two, the rest — Cavour, Victor Emmanuel, almost every minister in the 20th Century — look like fools. Even Mussolini seems to have bumbled his way into dictatorship.

Robinson Crusoe, Daniel Defoe

This is another book that I've read multiple times before, but probably not since I was twelve or so. I listened to big parts of it while dealing with some severe jet lag, waking up at 3am or so and then drifting back to sleep to the rhythms of narration about herding goats and defending against cannibals.

I was expecting this to be, shall we say... "un-modern" in its philosophy and tone seeing as how it's now exactly three centuries old. And while Friday's eager subservience no doubt puts this on many campus' Index Librorum Prohibitorum, I was actually surprised by the proto-moral-relativism and non-interference that Crusoe adopts toward the natives. There's also a ton of good lessons in here about the nature of wealth, desire, satisfaction, deserts, etc.

Cover of "The Book of Kells" by Bernhard Meehan

The Book of Kells, Bernard Meehan

I didn't read the Book of Kells itself since it's (a) in Latin and (b) so ornate it is nearly unreadable, but Meehan's book about that Book. It has wonderful illustrations throughout, and they are cross-referenced throughout the text in a very clever and unobtrusive way. The Book of Kells is one of the pinnacles of the early Western manuscript tradition, and I love it as much for its numerous imperfections as its elaborate decoration. As a calligrapher I do wish that Meehan talked a bit more about the letterforms as opposed to the illustrations, but I understand that's not something most people would be interested in.

I read this in advance of a trip to Ireland, where I went to see the exhibit of the original manuscript at Trinity College Dublin. I didn't pick this out for any particular reason, Meehan's book just happened to be the one book at my local library on the subject. It also happened to be the one book that the gift shop at Trinity was selling about The Book of Kells, so I suppose that means it's pretty authoritative?

by jsylvest Posted on 12 July 2019
Posted in Book List, Reviews | Tagged , , , , , , , , , , | Leave a comment

BaRT: Barrage of Random Transforms for Adversarially Robust Defense

This week I'm at CVPR — the IEEE's Computer Vision and Pattern Recognition Conference, which is a huge AI event. I'm currently rehearsing the timing of my talk one last time, but I wanted to take a minute between run-throughs to link to my co-author Steven Forsyth's wonderful post on the NVIDIA research blog about our paper.

Steven does a fantastic job of describing our work, so head over there to see what he has to say. I couldn't resist putting a post of my own because (a) I love this video we created...

...and (b), Steven left out what I think was the most convincing result we had, which shows that BaRT achieves a Top-1 accuracy on ImageNet that is higher than the Top-5 accuracy of the previous state-of-the-art defense, Adversarial Training.

A result from our paper, showing accuracy for varying adversarial distances.
Accuracy of BaRT under attack by PGD for varying adversarial distances, compared to the previous state-of-the-art.

Also, (c) I am very proud of this work. It's been an idea I've been batting around for almost three years now, and I finally got approval from my client to pursue it last year. It turns out it works exactly how I expected, and I can honestly say that this is the first — and probably only — time in my scientific career that has ever happened.

If you want a copy of the paper, complete with some code in the appendices, ((Our hands are somewhat tied releasing the full code due to the nature of our client relationship with the wonderful Laboratory for Physical Sciences, who funded this work.)) our poster, and the slides for our oral presentation you can find it on the BaRT page I slapped together on my website.

by jsylvest Posted on 19 June 2019
Posted in CS / Science / Tech / Coding | Tagged , , , , , | Leave a comment

Book List: 2019Q1

I think I did less reading this quarter than at any point since I beat dyslexia. Certainly less than any point since I started keeping track in 2011, and that includes the period when I finished my dissertation and had two kids. I'm teaching a course at a local college this semester, and lesson prep and grading has not left a lot of time for reading. But enough complaining...

slide:ology: The Art and Science of Creating Great Presentations, Nancy Duarte

Despite giving a fairly large number of presentations, I'm definitely not the audience for this. It's not really about presentations, but about sales presentations. If, like me, you have mostly factual & technical information to impart, I'm not sure how much this will help. There's a decent amount of advice in here if you're a complete graphic design novice, but there are probably better places to get that knowledge.

Cover of "The Relaxed Mind" by Dza Kilung
"The Relaxed Mind," Dza Kilung

The Relaxed Mind, Dza Kilung Rinpoche

There is perhaps a bit too much "woo" in the later chapters of this meditation manual, but it is still a good book for practice. If nothing else, I like having some meditation-related book on my bedstand/ipod: even if that book itself is not the best, it serves as an encouragement to keep practicing. The earlier two or three of the seven practices described here seem concretely useful. Maybe the latter practices will have more appeal to me as I become a "better" meditator?

Cover of "The Most Human Human," by Brian Christian
"The Most Human Human," Brian Christian

The Most Human Human: What Talking with Computers Teaches Us About What It Means to Be Alive, Brian Christian

I loved this. Christian has a degree in computer science and an MFA in poetry. I can't think of a better background to write about what the Turing Test tells us about talking with (and being) human. There's good history of AI, exploration of psychology and epistemology, and tips for what makes a conversation interesting.

I'm recommending this as a great book for other technologists to learn something about "soft skills" and for non-technologists to learn about AI. I can't think of another book that comes close to providing both benefits.

Lies Sleeping, Ben Aaronovitch

This is the latest in Aaronovitch's "Rivers of London" series, which I still love. I should really write these recaps as soon as I finish reading, because it's been long enough now that I don't have anything specific to say about it. But this is the ninth volume in the series, so if you don't already have an opinion about the prior eight, there's really no need for you to have one about this.

My wife, who reads mysteries almost exclusively, has recently started this series after hearing me talk about it since 2014. It's one of the few book series we both equally enjoy.

The Labyrinth Index, Charles Stross

(1) Copy-and-paste what I said above about not waiting to write these comments. (2) Copy-and-paste what I said about already having opinions about the series since it's long running, but replace "ninth" with "twelfth."

Cover of "Gravity's Rainbow" by Thomas Pynchon
"Gravity's Rainbow," Thomas Pynchon

Gravity's Rainbow, Thomas Pynchon

I'll be honest: I did not understand this book. I enjoyed it a great deal, but I did not understand it.

I like Pynchon as a stylist even when the narrative has me completely befuddled. As a result, even the confusing passages make for very good audiobook listening because I can let the language just wash over me.

Cover of "Zen Mind, Beginner's Mind" by Shunryu Suzuki.
"Zen Mind, Beginner's Mind," Shunryu Suzuki.

Zen Mind, Beginner's Mind, Shunryu Suzuki

I also didn't fully understand this book, but I feel like I wasn't really meant to. ((Actually, now that I think about it, maybe Pynchon didn't really want people to understand him either.)) I'm not sure "understanding" is even a thing you're supposed to be able to do to Zen. I think I got a lot out of it regardless. It's definitely something I'm going to revisit in the future.

Cover of "House of Suns" by Alastair Reynolds
"House of Suns," Alastair Reynolds

House of Suns, Alastair Reynolds

This is another winner. I haven't had this much fun reading a sci-fi book in years. It has that wide-screen baroque space opera feel that I used to get from Iain Banks books. I can't think of another story that engages so well with the sheer scope — in time and distance — of the galaxy. Before I was half way through I was already putting all of the library's other Reynolds books on my list.

The Sky-Blue Wolves, S. M. Stirling

I keep saying I'm going to stop reading this series, but then a new volume comes out just when I want a junk-food book and I read it anyway. Then I feel about as satisfied as I do after eating actual junk food. This is a fun world to mentally play around in, but Stirling is really phoning it in at this point. The Big Bad Guy that was supposed to require a world war to defeat just got knocked off in about a chapter of Dreamtime Ninja Shenanigans, and meanwhile two of our Intrepid Heroes (who happen to both be rightful heirs to continent-spanning empires) decided to have a love-child. Nice neat bow; everyone rides into the sunset.

by jsylvest Posted on 25 April 2019
Posted in Book List, Reviews | Tagged , , , , , , , | Leave a comment

Why we worry about the Ethics of Machine Intelligence

This essay was co-authored by myself and Steve Mills.

We worry about the ethics of Machine Intelligence (MI) and we fear our community is completely unprepared for the power we now wield. Let us tell you why.

To be clear, we’re big believers in the far-reaching good MI can do. Every week there are new advances that will dramatically improve the world. In the past month we have seen research that could improve the way we control prosthetic devices, detect pneumonia, understand long-term patient trajectories, and monitor ocean health. That’s in the last 30 days. By the time you read this, there will be even more examples. We really do believe MI will transform the world around us for the better, which is why we are actively involved in researching and deploying new MI capabilities and products.

There is, however, a darker side. MI also has the potential to be used for evil. One illustrative example is a recent study by Stanford University researchers who developed an algorithm to predict sexual orientation from facial images. When you consider recent news of the detainment and torturing of more than 100 male homosexuals in the Russian republic of Chechnya, you quickly see the cause for concern. This software and a few cameras positioned on busy street corners will allow the targeting of homosexuals at industrial-scale – hundreds quickly become thousands. The potential for this isn’t so far-fetched. China is already using CCTV and facial recognition software to catch jaywalkers. The researchers pointed out that their findings “expose[d] a threat to the privacy and safety of gay men and women.” That disavowal does little to prevent outside groups from implementing the technology for mass targeting and persecution.

Many technologies have the potential to be applied for nefarious purposes. This is not new. What is new about MI is the scale and magnitude of impact it can achieve. This scope is what will allow it to do so much good, but also so much bad. It is like no other technology that has come before. The notable exception being atomic weapons, a comparison others have already drawn. We hesitate to draw such a comparison for fear of perpetuating a sensationalistic narrative that distracts from this conversation about ethics. That said, it’s the closest parallel we can think of in terms of the scale (potential to impact tens of millions of people) and magnitude (potential to do physical harm).

None of this is why we worry so much about the ethics of MI. We worry because MI is unique in so many ways that we are left completely unprepared to have this discussion.

Ethics is not [yet] a core commitment in the MI field. Compare this with medicine where a commitment to ethics has existed for centuries in the form of the Hippocratic Oath. Members of the physics community now pledge their intent to do no harm with their science. In other fields ethics is part of the very ethos. Not so with MI. Compared to other disciplines the field is so young we haven’t had time to mature and learn lessons from the past. We must look to these other fields and their hard-earned lessons to guide our own behavior.

Computer scientists and mathematicians have never before wielded this kind of power. The atomic bomb is one exception; cyber weapons may be another. Both of these, however, represent intentional applications of technology.  While the public was unaware of the Manhattan Project, the scientists involved knew the goal and made an informed decision to take part. The Stanford study described earlier has clear nefarious applications; many other research efforts in MI may not. Researchers run the risk of unwittingly conducting studies that have applications they never envisioned and do not condone. Furthermore, research into atomic weapons could only be implemented by a small number of nation-states with access to proper materials and expertise. Contrast that with MI, where a reasonably talented coder who has taken some open source machine learning classes can easily implement and effectively ‘weaponize’ published techniques. Within our field, we have never had to worry about this degree of power to do harm. We must reset our thinking and approach our work with a new degree of rigor, humility, and caution.

Ethical oversight bodies from other scientific fields seem ill-prepared for MI. Looking to existing ethical oversight bodies is a logical approach. Even we suggested that MI is a “grand experiment on all of humanity” and should follow principals borrowed from human subject research. The fact that Stanford’s Institutional Review Board (IRB), a respected body within the research community, reviewed and approved research with questionable applications should give us all pause. Researchers have long raised questions about the broken IRB system. An IRB system designed to protect the interests of study participants may be unsuited for situations in which potential harm accrues not to the subjects but to society at large. It’s clear that the standards that have served other scientific fields for decades or even centuries may not be prepared for MI’s unique data and technology issues. These challenges are compounded even further by the general lack of MI expertise, or sometimes even technology expertise, within the members of these boards. We should continue to work with existing oversight bodies, but we must also take an active role in educating them and evolving their thinking towards MI.

MI ethical concerns are often not obvious. This differs dramatically from other scientific fields where ethical dilemmas are self-evident. That’s not to say they are easy to navigate. A recent story about an unconscious emergency room patient with a “Do Not Resuscitate” tattoo is a perfect example. Medical staff had to decide whether they should administer life-saving treatment despite the presence of the tattoo. They were faced with a very complex, but very obvious, ethical dilemma. The same is rarely true in MI where unintended consequences may not be immediately apparent and issues like bias can be hidden in complex algorithms. We have a responsibility to ourselves and our peers to be on the lookout for ethical issues and raise concerns as soon as they emerge.  

MI technology is moving faster than our approach to ethics. Other scientific fields have had hundreds of years for their approach to ethics to evolve alongside the science. MI is still nascent yet we are already moving technology from the ‘lab’ to full deployment. The speed at which that transition is happening has led to notable ethical issues including potential racism in criminal sentencing and discrimination in job hiring. The ethics of MI needs to be studied as much as the core technology if we ever hope to catch up and avoid these issues in the future. We need to catalyze an ongoing conversation around ethics much as we see in other fields like medicine, where there is active research and discussion within the community. 

The issue that looms behind all of this, however, is the fact that we can’t ‘put the genie back in the bottle’ once it has been released. We can’t undo the Stanford research now that it’s been published. As a community, we will forever be accountable for the technology that we create.

In the age of MI, corporate and personal values take on entirely new importance. We have to decide what we stand for and use that as a measure to evaluate our decisions. We can’t wait for issues to present themselves. We must be proactive and think in hypotheticals to anticipate the situations we will inevitably face.

Be assured that every organization will be faced with hard choices related to MI. Choices that could hurt the bottom line or, worse, harm the well-being of people now or in the future. We will need to decide, for example, if and how we want to be involved in Government efforts to vet immigrants or create technology that could ultimately help hackers. If we fail to accept that these choices inevitably exist, we run the risk of compromising our values. We need to stand strong in our beliefs and live the values we espouse for ourselves, our organizations, and our field of study. Ethics, like many things, is a slippery slope. Compromising once almost always leads to compromising again.

We must also recognize that the values of others may not mirror our own. We should approach those situations without prejudice. Instead of anger or defensiveness we should use them as an opportunity to have a meaningful dialog around ethics and values. When others raise concerns about our own actions, we must approach those conversations with humility and civility. Only then can we move forward as a community.

Machines are neither moral or immoral. We must work together to ensure they behave in a way that benefits, not harms, humanity. We don’t purport to have the answers to these complex issues. We simply request that you keep asking questions and take part in the discussion.

This has been crossposted to Medium and to the Booz Allen website as well.

We’re not the only one discussing these issues. Check out this Medium post by the NSF-Funded group Pervasive Data Ethics for Computational Research, Kate Crawford’s amazing NIPS keynote, Mustafa Suleyman’s recent essay in Wired UK, and Bryor Snefjella’s recent piece in BuzzFeed.

by jsylvest Posted on 23 March 2018
Posted in CS / Science / Tech / Coding | Tagged , , , , , , | Leave a comment

MalConv: Lessons learned from Deep Learning on executables

I don't usually write up my technical work here, mostly because I spend enough hours as is doing technical writing. But a co-author, Jon Barker, recently wrote a post on the NVIDIA Parallel For All blog about one of our papers on neural networks for detecting malware, so I thought I'd link to it here. (You can read the paper itself, "Malware Detection by Eating a Whole EXE" here.) Plus it was on the front page of Hacker News earlier this week, which is not something I thought would ever happen to my work.

Rather than rehashing everything in Jon's Parallel for All post about our work, I want to highlight some of the lessons we learned from doing this about ML/neural nets/deep learning.

As way of background, I'll lift a few paragraphs from Jon's introduction:

The paper introduces an artificial neural network trained to differentiate between benign and malicious Windows executable files with only the raw byte sequence of the executable as input. This approach has several practical advantages:

  • No hand-crafted features or knowledge of the compiler used are required. This means the trained model is generalizable and robust to natural variations in malware.
  • The computational complexity is linearly dependent on the sequence length (binary size), which means inference is fast and scalable to very large files.
  • Important sub-regions of the binary can be identified for forensic analysis.
  • This approach is also adaptable to new file formats, compilers and instruction set architectures—all we need is training data.

We also hope this paper demonstrates that malware detection from raw byte sequences has unique and challenging properties that make it a fruitful research area for the larger machine learning community.

One of the big issues we were confronting with our approach, MalConv, is that executables are often millions of bytes in length. That's orders of magnitude more time steps than most sequence processing networks deal with. Big data usually refers to lots and lots of small data points, but for us each individual sample was big. Saying this was a non-trivial problem is a serious understatement.

The MalConv architecture
Architecture of the malware detection network. (Image copyright NVIDIA.)

Here are three lessons we learned, not about malware or cybersecurity, but about the process of building neural networks on such unusual data.

1. Deep learning != image processing

The large majority of the work in deep learning has been done in the image domain. Of the remainder, the large majority has been in either text or speech. Many of the lessons, best practices, rules of thumb, etc., that we think apply to deep learning may actually be specific to these domains.

For instance, the community has settled around narrow convolutional filters, stacked with a lot of depth as being generally the best way to go. And for images, narrow-and-deep absolutely seems to be the correct choice. But in order to get a network that processes two million time steps to fit in memory at all (on beefy 16GB cards no less) we were forced to go wide-and-shallow.

With images, a pixel values is always a pixel value. 0x20 in a grayscale image is always darkish gray, no matter what. In an executable, a byte values are ridiculously polysemous: 0x20 may be part of an instruction, a string, a bit array, a compressed or encrypted values, an address, etc. You can't interpolate between values at all, so you can't resize or crop the way you would with images to make your data set smaller or introduce data augmentation. Binaries also play havoc with locality, since you can re-arrange functions in any order, among other things. You can't rely on any Tobbler's Law ((Everything is related, but near things are more related than far things.)) relationship the way you can in images, text, or speech.

2. BatchNorm isn't pixie dust

Batch Normalization has this bippity-boppity-boo magic quality. Just sprinkle it on top of your network architecture, and things that didn't converge before now do, and things that did converge now converge faster. It's worked like that every time I've tried it — on images. When we tried it on binaries it actually had the opposite effect: networks that converged slowly now didn't at all, no matter what variety of architecture we tried. It's also had no effect at all on some other esoteric data sets that I've worked on.

We discuss this at more length in the paper (§5.3), but here's the relevant figure:

BatchNorm activations
KDE plots of the convolution response (pre-ReLU) for multiple architectures. Red and orange: two layers of ResNet; green: Inception-v4; blue: our network; black dashed: a true Gaussian distribution for reference.

This is showing the pre-BN activations from MalConv (blue) and from ResNet (red & orange) and Inception-v4 (green). The purpose of BatchNorm is to output values in a standard normal, and it implicitly expects inputs that are relatively close to that. What we suspect is happening is that the input values from other networks aren't gaussian, but they're close-ish. ((I'd love to be able to quantify that closeness, but every test for normality I'm aware of doesn't apply when you have this many samples. If anyone knows of a more robust test please let me know.)) The input values for MalConv display huge asperity, and aren't even unimodal. If BatchNorm is being wonky for you, I'd suggest plotting the pre-BN activations and checking to see that they're relatively smooth and unimodal.

3. The Lump of Regularization Fallacy

If you're overfitting, you probably need more regularization. Simple advice, and easily executed. Everytime I see this brought up though, people treat regularization as if it's this monolithic thing. Implicitly, people are talking as if you have some pile of regularization, and if you need to fight overfitting then you just shovel more regularization on top. It doesn't matter what kind, just add more.

We ran in to overfitting problems and tried every method we could think of: weight decay, dropout, regional dropout, gradient noise, activation noise, and on and on. The only one that had any impact was DeCov, which penalized activities in the penultimate layer that are highly correlated with each other. I have no idea what will work on your data — especially if it's not images/speech/text — so try different types. Don't just treat regularization as a single knob that you crank up or down.

I hope some of these lessons are helpful to you if you're into cybersecurity, or pushing machine learning into new domains in general. We'll be presenting the paper this is all based on at the Artificial Intelligence for Cyber Security (AICS) workshop at AAAI in February, so if you're at AAAI then stop by and talk.

by jsylvest Posted on 8 December 2017
Posted in CS / Science / Tech / Coding | Tagged , , , , , , | Leave a comment

AI's "one trick pony" has a hell of a trick

The MIT Technology Review has a recent article by James Somers about error backpropagation, "Is AI Riding a One-Trick Pony?" Overall, I agree with the message in the article. We need to keep thinking of new paradigms because the SotA right now is very useful, but not correct in any rigorous way. However, as much as I agree with the thesis, I think Somers oversells it, especially in the beginning of the piece. For instance, the introductory segment concludes:

When you boil it down, AI today is deep learning, and deep learning is backprop — which is amazing, considering that backprop is more than 30 years old. It’s worth understanding how that happened—how a technique could lie in wait for so long and then cause such an explosion — because once you understand the story of backprop, you’ll start to understand the current moment in AI, and in particular the fact that maybe we’re not actually at the beginning of a revolution. Maybe we’re at the end of one.

That's a bit like saying "When you boil it down, flight is airfoils, and airfoils are Bernoulli's principle — which is amazing, considering that Bernoulli's principle is almost 300 years old." I totally endorse the idea that we ought to understand backprop; I've spent a lot of effort in the last couple of months organizing training for some of my firm's senior leadership on neural networks, and EBP/gradient descent is the heart of my presentation. But I would be very, very careful about concluding that backprop is the entire show.

Backprop was also not "lying in wait." People were working on it since it was introduced in 1986. The problem was that '86 was the height of the 2nd AI winter, which lasted another decade. Just like people should understand backprop to understand contemporary AI, they should learn about the history of AI to understand contemporary AI. Just because no one outside of CS (and precious few people in CS, for that matter) paid any attention to neural networks before 2015 doesn't mean they were completely dormant, only to spring up fully formed in some sort of intellectual Athenian birth.

I really don't want to be in the position of defending backprop. I took the trouble to write a dissertation about non-backprop neural nets for a reason, after all. ((That reason being, roughly put, that we're pretty sure the brain is not using backprop, and it seems ill-advised to ignore the mechanisms employed by the most intelligent thing we are aware of.)) But I also don't want to be in the position of letting sloppy arguments against neural nets go unremarked. That road leads to people mischaracterizing Minksy and Papert, abandoning neural nets for generations, and putting us epochs behind where we might have been. ((Plus sloppy arguments should be eschewed on the basis of the sloppiness alone, irrespective of their consequences.))

PS This is also worth a rejoinder:

Big patterns of neural activity, if you’re a mathematician, can be captured in a vector space, with each neuron’s activity corresponding to a number, and each number to a coordinate of a really big vector. In Hinton’s view, that’s what thought is: a dance of vectors.

That's not what thought is, that's how thought can be represented. Planets are not vectors, but their orbits can be profitably described that way, because "it behooves us to place the foundations of knowledge in mathematics." I'm sorry if that seems pedantic, but the distinction between a thing and its representation—besides giving semioticians something to talk about—underpins much of our interpretation of AI systems and cognitive science as well. Indeed, a huge chunk of data science work is figuring out the right representations. If you can get that, your problem is often largely solved. ((IIRC both Knuth and Torvalds have aphorisms to the effect that once you have chosen the correct data structures, the correct algorithms will naturally follow. I think AI and neuroscience are dealing with a lot of friction because we haven't been able to figure out the right representations/data structures. When we do, the right learning algorithms will follow much more easily.))

PPS This, on the other hand, I agree with entirely:

Deep learning in some ways mimics what goes on in the human brain, but only in a shallow way. … What we know about intelligence is nothing against the vastness of what we still don’t know.

What I fear is that people read that and conclude that artificial neural networks are built on a shallow foundation, so we should give up on them as being unreliable. A much better conclusion would be that we need to keep working and build better, deeper foundations.

by jsylvest Posted on 10 November 2017
Posted in CS / Science / Tech / Coding | Tagged , , , , , , , | Leave a comment

What I've Been Reading

Banana: The Fate of the Fruit That Changed the World, Dan Koeppel

Not bad. I'm a sucker for this type of history of a single commodity or common household object. It did make we want to try to get my hands on one of the few non-Cavendish cultivars of bananas that ever make their way to America.

Banana, Dan Koeppel

Very short summary/background info: all of the bananas at American grocery stores are genetic clones. This leaves them very susceptible to disease. This is not a theoretical concern: the variety which was previously ubiquitous was wiped out in the middle of the 20th century, and the current variety is being decimated in many growing regions. The fact that they're sterile fruits propagated clonally also makes it extremely difficult to breed other, more resistant varieties the way we do with the majority of our produce. Also important to the story is that banana production for the US and European markets has historically been very oligopolistic, leading to some … unsavory … business practices.

(Digression: The artificial banana flavor used in candies tastes nothing like our modern Cavendish bananas but I have heard that it is a very good match for the flavor of the Gros Michel that the Cavendish replaced. I've never been able to find definitive confirmation of that though, and I wish it was mentioned in this book. This a minor was disappointment. On the other hand, I was unreasonably amused by Koeppel's tongue-in-cheek translation of the "Gros Michel" as "The Big Mike".)

There is a temptation when people right books about subjects that have been overlooked to try to swing the pendulum too hard the other way to make the audience realize how important the subject is. Koeppel mostly avoids that trap, but falls in to it somewhat when discussing more current history. There's a lot more to Latin American politics and the rise of Bolivarian Socialism than the treatment of workers on banana plantations, which Koeppel asserted as a primary cause. Similarly he drew an analogy between the Clinton administration filing a complaint with the WTO about EU banana import duties and the active role that United Fruit played in shaping US foreign policy in Latin America between the beginning of the century and the middle of the Cold War. Both have fruit companies involved in international relations, that's where the similarities end. One of those is egregious cronyism, and one is a rules-based order in action.

Koeppel was on the shakiest ground towards the end of the book, when he was discussing the future of the banana market. His discussion of Fair Trade could benefit from reading Amrita Narlikar and similar critiques. I do give Koeppel much credit for his recognition of Consumer Sovereignty. If the conditions of banana growers are going to improve it won't be because Chiquita/Dole/etc. become kinder and gentler, it must be because consumers decide to spend more money buying bananas. Our stated preferences for better conditions for poor agricultural workers do not match our revealed preferences as consumers.

I also commend Koeppel for admitting that researching this book caused him to change his mind about transgenic food. He had previously been anti-GMO but became convinced genetic manipulation is the only way to save the banana crop. I do wish he had extended more of the same enthusiastic acceptance of transgenics to other crops, which he only went halfway to doing. Yes, bananas sterility makes them somewhat of a special case, but only somewhat.

A couple of months back I read Dan Reader's Potato: A History of the Propitious Esculent. If you're going to pick up one book about a non-cereal staple crop (and why wouldn't you?), I liked Potato much better.

Pilot X, Tom Merritt

This is a time-travel adventure story. It seemed like it could have been a Doctor Who episode. Merritt handles the oddities that result from time travel with deftness and wit. (A couple of examples that make you stop and think. 1: "This will be the last time I meet you, but not the last time you meet me." 2: The main character spends twelve years in training for a job, but it takes only four years, because when he gets to the end of the four year period he goes back in time and starts again twice, so that there are three of him all operating in parallel.) Amusing, but not great.

The Grace of Kings, Ken Liu

The Grace of Kings, Ken Liu

I had previous read Liu's short story collection The Paper Menagerie and loved it. Grace of Kings didn't disappoint. Highly recommended. One of the blurbs on the back cover described it as "the Wuxia version of Game of the Thrones" and that pretty much covers it.

One downside to the book is that characters experience rapid changes in fortune within the span of several pages. It's a nice, fast pace — most contemporary fantasy authors would lumberingly stretch out plot points like this for scores (hundreds?) of pages — but it does rob the story of some of the potential dramatic tension. One minute I've never even considered the possibility of a character rebelling against their overlord, and then within ten minutes they've plotted their rebellion, rebelled, been suppressed, and then punished. That doesn't give me much chance to savor the possibilities of what might happen. All in all though, I prefer this pace to the prolix plodding common to so many contemporary fantasy authors. I appreciate GRRM-style world building as much as the next reader, but not every fantasy novel needs every minor character to have their entire dynastic history spelled out, complete with descriptions of their heraldry, the architecture of their family seat, their favorite meals, and their sexual peccadilloes.

I'm not actually sure 'fantasy' is the correct term for Grace of Kings, come to think of it. There's some minor divine intervention and a couple of fantastic beasts, but no outright magic. I suppose it's fantasy in a sort of Homeric way, rather than a 20th century way.

Anyway, I've got my hands on the sequel, The Wall of Storms, and will be starting it as soon as possible. Hopefully we don't have to wait too long before the third and final volume is published.

The Art of War, Sun Tzu, translated by the Denma Translation Group

I listened to this audio edition from Shambhala Press. I don't pay much attention to which publishers produce which books, but I've been quite happy with several volumes of theirs that I've bought.

I hadn't read Art of the War in probably 20 years, so this was a good refresh. The way they structured this was to first have a recording of the text being read, and then start at the beginning with another reading but this time with interspersed commentary. That was a very good way to do it.

The text itself is short, and the commentary in this edition is good, so I'd recommend this even if you have read Art of War before.

The Map Thief, Michael Blanding

This is the story of an antiquities dealer specializing in rare maps, E. Forbes Smiley III, who turned to theft to build his inventory. I don't usually go for true crime stories, and indeed that was the least interesting aspect of this book. However, it was an interesting look at a little corner of the art/antiques market that I did not know about. There is also good information about the history of cartography, exploration and printing.((Everyone loves to hate on the Mercator projection, but this book does a good job of explaining how revolutionarily useful Mercator's cartography was in the 16th century. His projection is a tool that is remarkably good for its intended purpose, i.e. helping navigate over long sea voyages. It shouldn't be used the way it has been (hung on every classroom wall, making choropleth infographics, etc.) but that doesn't make it a bad tool per se, just one that is misused. The fitness of a technology, just like that of a biological organism, can only be usefully evaluated in the environment it is adapted for.))

Perhaps the most interesting part of the case for me came after Smiley confessed and the various libraries he stole from had to go about figuring out what was missing and from whom. In the case of normal thefts, or even art thefts, this is pretty straight forward, but the nature of the material — rare, poorly or partially catalogued, incompletely and idiosyncraticly described, existing in various editions with only marginal differences, etc. — make it quite a puzzle. Coming up with a good cataloging system for oddities like antique maps would make a good exercise for a library science/information systems/database project. (Indeed it was only thanks to the work of a former Army intelligence analyst that things got sorted out as well as they did.) Even something superficially simple like figuring out which copy of a printed map is which makes for a good computer vision challenge.

There are also Game Theoretic concerns at work: libraries would benefit if they all operated together to publicize thefts in order to track down stolen materials, but it is in every individual library's interest to cover up thefts so as not to besmirch their reputation and alienate donors, who expect that materials they contribute will be kept safe. The equilibrium is not straightforward, nor is it likely to be optimal.

by jsylvest Posted on 25 July 2017
Posted in Book List, Reviews | Tagged , , | Leave a comment

Reading List for 23 September 2013

Arnold Kling :: Big Gods

Here is a question to think about. If religions help to create social capital by allowing people to signal conscientiousness, conformity, and trustworthiness [as Norenzayan claims], how does this relate to Bryan Caplan’s view that obtaining a college degree performs that function?

That might explain why the credentialist societies of Han China were relatively irreligious. Kling likes to use the Vickies/Thetes metaphor from Neal Stephenson's Diamond Age, and I think this dichotomy could play well with that. Wouldn't the tests required by the Reformed Distributed Republic fill this role, for instance?

Ariel Procaccia :: Alien journals

Steve Landsburg :: RIP, Ronald Coase

This is by far the best, simplest explanation of Coase's insights that I have read. Having read plenty of Landsburg, that should not — indeed does not — surprise me.

His final 'graph is a digression, but a good point:

Coase’s Nobel Prize winning paper is surely one of the landmark papers of 20th century economics. It’s also entirely non-technical (which is fine), and (in my opinion) ridiculously verbose (which is annoying). It’s littered with numerical examples intended to illustrate several different but related points, but the points and the examples are so jumbled together that it’s often difficult to tell what point is being illustrated... Pioneering work is rarely presented cleanly, and Coase was a true pioneer.

And this is why I put little stock in "primary sources" when it comes to STEM. The intersection between people/publications who originate profound ideas and people/publications which explain profound ideas well is a narrow one. If what you want is the latter, don't automatically mistake it for the former. The best researchers are not the best teachers, and this is true as much for papers as it is for people.

That said, sometimes the originals are very good. Here are two other opinions on this, from Federico Pereiro and John Cook.

Prosthetic Knowledge :: Prototypo.io

Start a font by tweaking all glyphs at once. With more than twenty parameters, design custom classical or experimental shapes. Once prototyping of the font is done, each point and curve of a glyph can be easily modified. Explore, modify, compare, export with infinite variations.

I liked this better when it was called Metafont.

Sorry, I couldn't resist some snark. I actually do like this project. I love both Processing and typography, so why wouldn't I? Speaking of which...

Hoefler & Frere-Jones :: Pilcrow & Capitulum

Some sample pilcrows from the H&FJ foundry.
Some sample pilcrows from the H&FJ foundry.

Eric Pement :: Using SED to make indexes for books

That's some impressive SED-fu.

Mike Duncan :: Revolutions Podcast

(Okay, so technically this may not belong on a "reading list.") Duncan previously created The History of Rome podcast, which is one of my favorites. Revolutions is his new project, and it just launched. Get on board now.

Kenneth Moreland :: Diverging Color Maps for Scientific Visualization [pdf]

Ardi, Tan & Yim :: Color Palette Generation for Nominal Encodings [pdf]

These two have been really helpful in the new visualization project I'm working on.

Andrew Shikiar :: Predicting Kiva Loan Defaults

Brett Victor :: Up and Down the Ladder of Abstraction: A Systematic Approach to Interactive Visualization

This would be a great starting place for high-school or freshmen STEM curricula. As a bonus, it has this nice epigraph from Richard Hamming:

"In science, if you know what you are doing, you should not be doing it. In engineering, if you do not know what you are doing, you should not be doing it. Of course, you seldom, if ever, see either pure state."

Megan McArdle :: 13 Tips for Jobless Grads on Surviving the Basement Years

I'm at the tail end of a doctoral program and going on the job market. This is good advice. What's disappointing is that this would have been equally good and applicable advice for people going on the job market back when I started grad school. The fact that we're five years (!!) down the road and we still have need of these sorts of "surviving in horrid job markets" pieces is bleak.

by jsylvest Posted on 23 September 2013
Posted in Reading Lists | Tagged , , , , | Leave a comment

Reading List for 16 July 2013

Evan Miller :: Winkel Tripel Warping Trouble or "How I Found a Bug in the Journal of Surveying Engineering"

All programming blogs need at least one post unofficially titled “Indisputable Proof That I Am Awesome.” These are usually my favorite kind of read, as the protagonist starts out with a head full of hubris, becomes mired in self-doubt, struggles on when others would have quit, and then ultimately triumphs over evil (that is to say, slow or buggy computer code), often at the expense of personal hygiene and/or sanity.

I'm a fan of the debugging narrative, and this is a fine example of the genre. I've been wrestling with code for mapping projections recently, so I feel Miller's pain specifically. In my opinion the Winkel Tripel is mathematically gross, but aesthetically unsurpassed. Hopefully I'll find some time in the next week or so to put up a post about my mapping project.

Irene Global Tweets WInkel Tripel
A screenshot of a project I've been working on to map geotagged tweets.

Kevin Grier :: Breaking down the higher ed wage premium

wage premium by major
Wage premium and popularity of majors

File under "all college degrees are not created equal" or perhaps "no, junior, you may not borrow enough to buy a decent house in order to get a BA in psych."

Aleatha Parker-Wood :: One Shot vs Iterated Games

Social cohesion can be thought of as a manifestation of how "iterated" people feel their interactions are, how likely they are to interact with the same people again and again and  have to deal with long term consequences of locally optimal choices, or whether they feel they can "opt out" of consequences of interacting with some set of people in a poor way.

Mike Munger :: Grade Inflation? Some data

Munger links to some very good analysis but it occurs to me that what is really needed is the variance of grades over time and not just the mean. (Obviously these two things are related since the distribution is bounded by [0, 4]. A mean which has gone from 2.25 to 3.44 will almost certainly result in less variance here.)

I don't much care where the distribution is centered. I care how wide the distribution is — that's what lets observers distinguish one student from another. Rankings need inequality. Without it they convey no information.

Marginal Revolution :: Alex Tabarrok :: The Battle over Junk DNA

I share Graur's and Tabarrok's wariness over "high impact false positives" in science. This is a big problem with no clear solutions.

The Graur et al. paper that Tabarrok discusses is entertaining in its incivility. Sometimes civility is not the correct response to falsehoods. It's refreshing to see scientists being so brutally honest with their opinions. Some might say they are too brutal, but at least they've got the honest part.

Peter McCaffrey :: 5 reasons price gouging should be legal: Especially during disasters

McCaffrey is completely right. But good luck to him reasoning people out of an opinion they were never reasoned into in the first place.

I do like the neologism "sustainable pricing" that he introduces. Bravo for that.

I would add a sixth reason to his list: accusations of "price gouging" are one rhetorical prong in an inescapable triple bind. A seller has three MECE choices: price goods higher than is common, the same as is common, or lower than is common. These choices will result in accusations of price gouging, collusion, and anti-competitive pricing, respectively. Since there is no way to win when dealing with people who level accusations of gouging, the only sensible thing to do is ignore them.

Shawn Regan :: Everyone calm down, there is no “bee-pocalypse”

Executive summary: apiarists have agency, and the world isn't static. If the death rate of colonies increases, they respond by creating more colonies. Crisis averted.

Eliezer Yudkowsky :: Betting Therapy

"Betting Therapy" should be a thing. You go to a betting therapist and describe your fears — everything you're afraid will happen if you do X — and then the therapist offers to bet money on whether it actually happens to you or not. After you lose enough money, you stop being afraid.

Sign me up.

by jsylvest Posted on 16 July 2013
Posted in Reading Lists | Tagged , , , , , , | Leave a comment