Massassi Forums Logo

This is the static archive of the Massassi Forums. The forums are closed indefinitely. Thanks for all the memories!

You can also download Super Old Archived Message Boards from when Massassi first started.

"View" counts are as of the day the forums were archived, and will no longer increase.

☖ Forums → Discussion Forum → Computer Science and Math and Stuff
1234567891011121314151617181920212223242526272829303132333435
Computer Science and Math and Stuff
2017-12-30, 12:17 AM #1
I thought since so many of us are in these areas we could have a thread about such subjects.

In my boredom today I launched up a WoW client and began prodding for exploits. I haven't found anything useful yet but I made the server kinda angry when forcing in negative values for auctions. Way to blow a few hours I guess.

What are yall doing with your time?
2017-12-30, 2:04 AM #2
I made a Blaze clone.

The key difference for mine is that it does some stuff with test coverage. Normally test coverage is not a useful metric: code is often exercised as a side effect during a test, so test coverage will report that some region of code is tested even if there aren’t any tests that specifically exercise it. That means test coverage is usually only a good metric for testability, in the very narrow sense that it’s possible to write a test that can touch everything in your code (even if you can’t intelligently assert anything about it).

In my Blaze clone, I’ve added test annotations. They’re special comments that say which tests exercise different regions of source code, which you manually add to your code as you write and test it. When you run your tests, the coverage data is filtered against those annotations. So, for example, if you say that your class foo is tested by all unit tests named foo_test_*, then all coverage data is ignored except that produced by tests named foo_test_*.

The idea is to make it easy for a well-meaning user to detect which code has just miserably inadequate testing. Obviously it’s not a policy enforcement tool; there’s nothing stopping someone from saying their code is covered by all tests. But if you’re writing code and unit tests at the same time, and you want to make sure your tests are covering the code you mean them to cover, this tool can tell you for sure.

It also needed to be fast, to make this workflow productive. So, well, it’s a Blaze clone, and that helped. Per tradition, there’s a daemon caching a bunch of stuff in memory between builds. I’ve also rigged up gcov to send over data through shared memory, so it never has to touch disk. I also only parse the gcov data files that are needed, per the annotations.

Really the most important thing will be the next step, which is adding a way to automatically re-run exactly the tests that are needed to make the filtered coverage data up-to-date. It’s not a substitute for a full test run, by any means, but it will speed up local test development and that is the priority.

I mean, this is a neat and useful diversion for me, but the real goal was to make a build system with automatic dependency license compliance. lol. It’s in my backlog.


Related to this, some months ago I did a bunch of work setting up tooling and CI. I have it set up so that each commit has to pass through Phabricator. Each commit must get sign-off from CI build, test, and static analysis, before it can get merged to master. (If I weren’t working by myself it would also need a code review from a real person, and not just Jenkins.) It wasn’t easy, but Phabricator made this workflow possible. I can tell you from personal experience that trying to do this with Jenkins alone is a painful journey.
2017-12-31, 6:23 PM #3
Originally posted by Jon`C:
I made a Blaze clone.


Just to be clear: we are talking about this Blaze?

Originally posted by Jon`C:
The key difference for mine is that it does some stuff with test coverage. Normally test coverage is not a useful metric: code is often exercised as a side effect during a test, so test coverage will report that some region of code is tested even if there aren’t any tests that specifically exercise it. That means test coverage is usually only a good metric for testability, in the very narrow sense that it’s possible to write a test that can touch everything in your code (even if you can’t intelligently assert anything about it).

In my Blaze clone, I’ve added test annotations. They’re special comments that say which tests exercise different regions of source code, which you manually add to your code as you write and test it. When you run your tests, the coverage data is filtered against those annotations. So, for example, if you say that your class foo is tested by all unit tests named foo_test_*, then all coverage data is ignored except that produced by tests named foo_test_*.

The idea is to make it easy for a well-meaning user to detect which code has just miserably inadequate testing. Obviously it’s not a policy enforcement tool; there’s nothing stopping someone from saying their code is covered by all tests. But if you’re writing code and unit tests at the same time, and you want to make sure your tests are covering the code you mean them to cover, this tool can tell you for sure.


So from what I understand from reading Wikipedia: test coverage is a simple function which returns what percentage of code is actually run for a given set of inputs. What you did was, add to Blaze test coverage that can be labelled and run on specific blocks of code, which gives more accurate information about where the code is that's being run or not run.

Is that right?

Originally posted by Jon`C:
It also needed to be fast, to make this workflow productive. So, well, it’s a Blaze clone, and that helped. Per tradition, there’s a daemon caching a bunch of stuff in memory between builds. I’ve also rigged up gcov to send over data through shared memory, so it never has to touch disk. I also only parse the gcov data files that are needed, per the annotations.


So, again trying to read from Wikipedia, gcov keeps track of the amount of times statements are called. Since I'm definitely not fully understanding what exactly you're doing, let me ask a stupid question: isn't test coverage calculable from the output of gcov? It seems you could do some simple arithmetic to find out what percentage of statements were called based solely on gcov data.

But, Blaze seems to be "higher level" analytics on software usage, so is what you're doing packaging together some tools that give efficiency results on different levels?

Originally posted by Jon`C:
Really the most important thing will be the next step, which is adding a way to automatically re-run exactly the tests that are needed to make the filtered coverage data up-to-date. It’s not a substitute for a full test run, by any means, but it will speed up local test development and that is the priority.

I mean, this is a neat and useful diversion for me, but the real goal was to make a build system with automatic dependency license compliance. lol. It’s in my backlog.


So it's like: you can break down the code into various sections, identify which sections are inefficient, then update that code and simply run tests on that one section as you update? So that the tests are targetted and more efficient.

Originally posted by Jon`C:
Related to this, some months ago I did a bunch of work setting up tooling and CI. I have it set up so that each commit has to pass through Phabricator. Each commit must get sign-off from CI build, test, and static analysis, before it can get merged to master. (If I weren’t working by myself it would also need a code review from a real person, and not just Jenkins.) It wasn’t easy, but Phabricator made this workflow possible. I can tell you from personal experience that trying to do this with Jenkins alone is a painful journey.


So Jenkins seems to take whatever code you've worked on, and automatically attempts to integrate/compile them and return error messages so you get updates on how your changes are effecting the project in real-time? I guess what you're doing then, is integrating your Blaze clone into Jenkins to automate messaging about the test coverage/efficiency of specific blocks of code as you work on them.

Sorry my replies can't be more thoughtful. I have very little experience in managing workflow or dealing with large software projects, so I'm trying to wrap my mind around the issues you're dealing with and how you're trying to solve them. In fact, I should probably read up on using Git, and more about like developing software in this way.
2017-12-31, 6:31 PM #4
Originally posted by Reid:
I thought since so many of us are in these areas we could have a thread about such subjects.

In my boredom today I launched up a WoW client and began prodding for exploits. I haven't found anything useful yet but I made the server kinda angry when forcing in negative values for auctions. Way to blow a few hours I guess.

What are yall doing with your time?


So update on my "hacks for WoW 2.4.3" fun project: Just slicing through the UI and forcing it to send commands to the server doesn't seem fruitful. So, instead, I've started packet sniffing. However, despite spending a bit of time filtering through captured packets, it seems the contents of many (but not the headers?) of the packets are encrypted. Some aren't, but some are, but the point is, none of the commands I'm sending via the auction house appear to be unencrypted, unless the protocol is doing something weird.

So, instead, I'm writing some code casually to hook various WinSock functions, and then I'll try to locate what parts of the assembly are calling it. Hopefully any encryption/obfuscation code will be easy enough to hook and intercept/modify packets going to the server. From there I'll have to locate AH opcodes and start dinking around with packets. This method is probably preferred to simply cutting up the UI functions, and I have a feeling will expose more potential exploits.
2017-12-31, 6:35 PM #5
haha, sorry. By Blaze I mean Google Blaze, their internal build system. Major open source relatives are Bazel, Buck, Pants.
2017-12-31, 6:36 PM #6
Also, today I noticed the same thing in two unrelated locations, which had me pondering. The two things:

DEF CON 25 (2017) - Weaponizing Machine Learning - Petro, Morris

Section from Intro to Theoretical Computer Science

From the theoretical computer science thing I linked:

Quote:
Before explaining the math background, perhaps I should explain why does this course is so “mathematically heavy”. After all, this is supposed to be a course about computation; one might think we should be talking mostly about programs, rather than more “mathematical” objects such as sets, functions, and graphs, and doing more coding on an actual computer than writing mathematical proofs with pen and paper. So, why are we doing so much math in this course? Is it just some form of hazing? Perhaps a revenge of the “math nerds” against the “hackers”?


The sentiment was also expressed in the video, which expressed disdain for mathematics as being obfuscatory or otherwise hard to penetrate. Is this sentiment really that common among computer scientists? I'll admit that, in my limited experiences with undergraduates, I didn't think highly of their ability to reason mathematically™ (in the proper sense), although many had some competency in math. But the thing is, the topics the Intro are decrying as "too mathy" are hardly more difficult than lower division or intro upper division math topics, and not particularly challenging ones at that. To me, it feels frustrating that frankly elementary topics should be treated as soooo hard by CS people. Not sure why this is, honestly.
2017-12-31, 6:41 PM #7
Originally posted by Jon`C:
haha, sorry. By Blaze I mean Google Blaze, their internal build system. Major open source relatives are Bazel, Buck, Pants.


So this is about optimizing the time it takes to build your code.
2017-12-31, 8:34 PM #8
Originally posted by Reid:
So from what I understand from reading Wikipedia: test coverage is a simple function which returns what percentage of code is actually run for a given set of inputs.
There are really two ways people talk about test coverage. The first refers to the metric you've described here, which in simpler terms is the percentage of code executed during tests. The second way refers to the actual instrumentation data used to calculate the metric, which tells you specifically what code was and was not executed during tests.

I don't use the former, and I will never talk about it outside of this context. It doesn't tell you what management types think it does, and what it does tell you isn't useful for very much.

Quote:
What you did was, add to Blaze test coverage that can be labelled and run on specific blocks of code, which gives more accurate information about where the code is that's being run or not run.

Is that right?
Not quite. I added a way to mask your source code, so it automatically ignores coverage data from a test unless the source code explicitly says that it's tested by that test.

It gives more accurate information about whether the part of code you're working on is tested by a test for that code, as opposed to a test for some other code.

Quote:
So, again trying to read from Wikipedia, gcov keeps track of the amount of times statements are called.
Arcs! Gcov tracks the number of times arcs are followed, not statements executed. It instruments branch and jump instructions. After testing you use a lookup table to convert the arc counts (profile data) into branch coverage, block coverage, or whatever lower form of coverage you want.

Gcov has no way of producing statement coverage. I'm using line coverage, which is approximately similar as far as an end user is concerned. Line coverage isn't as rigorous as branch coverage, but it's easier to visualize and my existing tools support it. That is the most important thing for me.

Quote:
Since I'm definitely not fully understanding what exactly you're doing, let me ask a stupid question: isn't test coverage calculable from the output of gcov? It seems you could do some simple arithmetic to find out what percentage of statements were called based solely on gcov data.
The metric, yes.

Quote:
So Jenkins seems to take whatever code you've worked on, and automatically attempts to integrate/compile them and return error messages so you get updates on how your changes are effecting the project in real-time?
This is more or less accurate. Continuous integration is more important when you have multiple people working on a project at the same time, so your work is constantly being compiled and tested against theirs. Generally, the faster you find an issue, the cheaper it is to fix. A nice side effect of continuous integration is that you're always ready to cut a release (assuming you are doing it properly).

As a solo developer, I mostly use Jenkins for scheduled integration testing and coordinating builds across multiple platforms. Basically it means I can spend most of my time developing exclusively on Linux, and I don't have to worry about missing a problem with the Windows version because Jenkins will automatically tell me.

Quote:
I guess what you're doing then, is integrating your Blaze clone into Jenkins to automate messaging about the test coverage/efficiency of specific blocks of code as you work on them.
Jenkins uses my Blaze clone to produce a report, yes. Commits are gated on test suite adequacy.

Quote:
I have very little experience in managing workflow or dealing with large software projects, so I'm trying to wrap my mind around the issues you're dealing with and how you're trying to solve them. In fact, I should probably read up on using Git, and more about like developing software in this way.
You absolutely should. Development tooling is a force multiplier.

https://mike-bland.com/2011/12/02/coding-and-testing-at-google-2006-vs-2011.html

Originally posted by Reid:
The sentiment was also expressed in the video, which expressed disdain for mathematics as being obfuscatory or otherwise hard to penetrate. Is this sentiment really that common among computer scientists? I'll admit that, in my limited experiences with undergraduates, I didn't think highly of their ability to reason mathematically™ (in the proper sense), although many had some competency in math. But the thing is, the topics the Intro are decrying as "too mathy" are hardly more difficult than lower division or intro upper division math topics, and not particularly challenging ones at that. To me, it feels frustrating that frankly elementary topics should be treated as soooo hard by CS people. Not sure why this is, honestly.
Bad experiences. Started writing about it, but decided I don't want to talk about it yet. Suffice to say, it's not just a problem with math. They do this about everything.
2017-12-31, 8:45 PM #9
Originally posted by Reid:
So this is about optimizing the time it takes to build your code.


Optimize and time are vague terms, so I generally try not to use them. It's also not just about builds.

A build system is really a frontend for your local development workflow. This is about shortening my build-test loop. That doesn't just involve a fast build, although that is a part of it. It also means writing tools and exposing the right user interface to enable more efficient behaviors.
2017-12-31, 8:54 PM #10
Originally posted by Reid:
Also, today I noticed the same thing in two unrelated locations, which had me pondering. The two things:

DEF CON 25 (2017) - Weaponizing Machine Learning - Petro, Morris

Section from Intro to Theoretical Computer Science

From the theoretical computer science thing I linked:



The sentiment was also expressed in the video, which expressed disdain for mathematics as being obfuscatory or otherwise hard to penetrate. Is this sentiment really that common among computer scientists? I'll admit that, in my limited experiences with undergraduates, I didn't think highly of their ability to reason mathematically™ (in the proper sense), although many had some competency in math. But the thing is, the topics the Intro are decrying as "too mathy" are hardly more difficult than lower division or intro upper division math topics, and not particularly challenging ones at that. To me, it feels frustrating that frankly elementary topics should be treated as soooo hard by CS people. Not sure why this is, honestly.


Ever read a formal proof of correctness of a computer program?
2017-12-31, 9:13 PM #11
Originally posted by Jon`C:
It also means writing tools and exposing the right user interface to enable more efficient behaviors.
I'll put it this way.

You're working on a team at a medium-sized company. Your build is all pretty clean Antfiles or Makefiles, maybe some of your projects have even been flirting with Gradle. You've got three phony targets: all, test, and clean, no more, no less, just as the good Lord intended. You've got a powerful workstation for running local builds and tests. You have Jenkins set up to fire off a build and test run every time someone pushes code. Every couple of days you run static analysis, and every few days you collect test coverage, and someone is assigned to read those reports. Some of you choose to do pair programming, some of you choose to do code reviews/pull requests. If that's what your company looks like, you're doing just fine. You're probably better off than 95% of companies out there.

But here's the thing.

Google doesn't have clean Makefiles or Antfiles, they have a build system with a well-specified, queryable, partition-tolerant dependency graph. Google doesn't give developers a powerful workstation, they've got a build system that can dispatch builds and tests to an entire server farm. Google doesn't fire off a build and test every time someone lands code, they do it before the code is allowed to land. They don't lint every few days, and they don't get coverage every few days, they do it for every commit, and that information is included in every code review.

All of these things are well within the reach of any organization. There's basically no excuse not to be doing this stuff. The productivity advantages of adequate testing and code reviews is settled science at this point, and there are many software tools that can help with policy enforcement.

And, y'know, they'll say things like "we can never compete against google anyway, so we don't care about Google's productivity". Yeah, but if you aren't doing this stuff, you also won't get bought by Google. You can't even try to compete against a huge company, but you sure can compete against them on development costs. If Google has an 80% productivity advantage over you, it'll literally never even be worth their time to talk to you before they destroy you.
2018-01-02, 9:42 PM #12
Fer frick's sake.

Quote:
A fundamental design flaw in Intel's processor chips has forced a significant redesign of the Linux and Windows kernels to defang the chip-level security bug.

Programmers are scrambling to overhaul the open-source Linux kernel's virtual memory system. Meanwhile, Microsoft is expected to publicly introduce the necessary changes to its Windows operating system in an upcoming Patch Tuesday: these changes were seeded to beta testers running fast-ring Windows Insider builds in November and December.

Crucially, these updates to both Linux and Windows will incur a performance hit on Intel products. The effects are still being benchmarked, however we're looking at a ballpark figure of five to 30 per cent slow down, depending on the task and the processor model.
Specifically, the performance cost of the fix depends on how much of your workload involves system calls. Syscalls are how a program talks to your operating system. Lots of programs don't need to do this much, so they won't be slowed down. Games in particular make a lot of syscalls.

Quote:
It is understood the bug is present in modern Intel processors produced in the past decade. It allows normal user programs – from database applications to JavaScript in web browsers – to discern to some extent the layout or contents of protected kernel memory areas.
Ed: completely discern. The handwavy explanation is that Intel is ~2fast4security~ and didn't do anything to prevent speculative execution of reads from protected memory in unprivileged processes. If the kernel had any secrets, they've been exposed to any process running on the same machine.

Quote:
The fix is to separate the kernel's memory completely from user processes using what's called Kernel Page Table Isolation, or KPTI. At one point, Forcefully Unmap Complete Kernel With Interrupt Trampolines, aka ****WIT, was mulled by the Linux kernel team, giving you an idea of how annoying this has been for the developers.
All mainstream operating systems map the kernel into every process address space for performance reasons. OSes can't do that anymore, at least not until every current Intel CPU is out of use.

AMD CPUs aren't affected.

https://www.theregister.co.uk/2018/01/02/intel_cpu_design_flaw/
2018-01-02, 9:49 PM #13
I wonder if this affects Xen, or if its virtualization already incurs this kind of performance hit anyway when system calls are used.

Edit:
Quote:
There were rumors of a severe hypervisor bug – possibly in Xen – doing the rounds at the end of 2017. It may be that this hardware flaw is that rumored bug: that hypervisors can be attacked via this kernel memory access cockup, and thus need to be patched, forcing a mass restart of guest virtual machines.
2018-01-02, 10:05 PM #14
Originally posted by Reverend Jones:
I wonder if this affects Xen, or if its virtualization already incurs this kind of performance hit anyway when system calls are used.


I don't know about Xen's virtualization technology specifically, but given that Amazon and Google are both involved it seems likely that VPSes are affected by the security bug.
2018-01-03, 1:29 AM #15
Originally posted by Jon`C:
There are really two ways people talk about test coverage. The first refers to the metric you've described here, which in simpler terms is the percentage of code executed during tests. The second way refers to the actual instrumentation data used to calculate the metric, which tells you specifically what code was and was not executed during tests.

I don't use the former, and I will never talk about it outside of this context. It doesn't tell you what management types think it does, and what it does tell you isn't useful for very much.


Right, it seems straightforwardly sensible that knowing where the code is more efficient matters more than how much it's inefficient.

Originally posted by Jon`C:
Not quite. I added a way to mask your source code, so it automatically ignores coverage data from a test unless the source code explicitly says that it's tested by that test.

It gives more accurate information about whether the part of code you're working on is tested by a test for that code, as opposed to a test for some other code.


Gotcha, so if a particular test fails you can get more accurate information about where in your code the bug lies.

Originally posted by Jon`C:
Arcs! Gcov tracks the number of times arcs are followed, not statements executed. It instruments branch and jump instructions. After testing you use a lookup table to convert the arc counts (profile data) into branch coverage, block coverage, or whatever lower form of coverage you want.

Gcov has no way of producing statement coverage. I'm using line coverage, which is approximately similar as far as an end user is concerned. Line coverage isn't as rigorous as branch coverage, but it's easier to visualize and my existing tools support it. That is the most important thing for me.


I'm guessing GNU builds the lookup table. Without a kind of Perforce setup, does the extra time it takes to build the table significant? My intuition says probably not.

Originally posted by Jon`C:
This is more or less accurate. Continuous integration is more important when you have multiple people working on a project at the same time, so your work is constantly being compiled and tested against theirs. Generally, the faster you find an issue, the cheaper it is to fix. A nice side effect of continuous integration is that you're always ready to cut a release (assuming you are doing it properly).

As a solo developer, I mostly use Jenkins for scheduled integration testing and coordinating builds across multiple platforms. Basically it means I can spend most of my time developing exclusively on Linux, and I don't have to worry about missing a problem with the Windows version because Jenkins will automatically tell me.


Yep, and I think after reading the article you linked I think I understand more clearly the point of these various tools. Actually might try to set up some stuff with g++ and get more into the tooling side of programming rather than the "finish projects that compile" since the world seems intent on sucking me into programming work like a black hole.
2018-01-03, 1:35 AM #16
Originally posted by Jon`C:
Optimize and time are vague terms, so I generally try not to use them. It's also not just about builds.

A build system is really a frontend for your local development workflow. This is about shortening my build-test loop. That doesn't just involve a fast build, although that is a part of it. It also means writing tools and exposing the right user interface to enable more efficient behaviors.


Yeah, I think I see that now. The tools are going to "slow down" development time in some sense, and I think my previous thoughts were latching more onto the increased amount of computation needed, but again that article made it clear in which ways the tools are cutting down lost time by effectively making it much easier and quick to spot and rollback code that's not working.

Originally posted by Jon`C:
Bad experiences. Started writing about it, but decided I don't want to talk about it yet. Suffice to say, it's not just a problem with math. They do this about everything.


Fun, well if people in programming refuse to learn math, that increases the value of skills in math.

Originally posted by Reverend Jones:
Ever read a formal proof of correctness of a computer program?


No, I haven't. I have seen very little in theoretical computer science that's not related to geometric group theory, I'm afraid.
2018-01-03, 3:13 AM #17
Originally posted by Jon`C:
I'll put it this way.

You're working on a team at a medium-sized company. Your build is all pretty clean Antfiles or Makefiles, maybe some of your projects have even been flirting with Gradle. You've got three phony targets: all, test, and clean, no more, no less, just as the good Lord intended. You've got a powerful workstation for running local builds and tests. You have Jenkins set up to fire off a build and test run every time someone pushes code. Every couple of days you run static analysis, and every few days you collect test coverage, and someone is assigned to read those reports. Some of you choose to do pair programming, some of you choose to do code reviews/pull requests. If that's what your company looks like, you're doing just fine. You're probably better off than 95% of companies out there.

But here's the thing.

Google doesn't have clean Makefiles or Antfiles, they have a build system with a well-specified, queryable, partition-tolerant dependency graph. Google doesn't give developers a powerful workstation, they've got a build system that can dispatch builds and tests to an entire server farm. Google doesn't fire off a build and test every time someone lands code, they do it before the code is allowed to land. They don't lint every few days, and they don't get coverage every few days, they do it for every commit, and that information is included in every code review.

All of these things are well within the reach of any organization. There's basically no excuse not to be doing this stuff. The productivity advantages of adequate testing and code reviews is settled science at this point, and there are many software tools that can help with policy enforcement.

And, y'know, they'll say things like "we can never compete against google anyway, so we don't care about Google's productivity". Yeah, but if you aren't doing this stuff, you also won't get bought by Google. You can't even try to compete against a huge company, but you sure can compete against them on development costs. If Google has an 80% productivity advantage over you, it'll literally never even be worth their time to talk to you before they destroy you.


So like, if you're going to run your own company, set up a server farm with Perforce and develop a ton of tools to compete with Google? I mean, sounds nice, but aren't the barriers to achieve that sort of workflow high and wouldn't it take a long time to develop?
2018-01-03, 3:17 AM #18
Originally posted by Reid:
Right, it seems straightforwardly sensible that knowing where the code is more efficient matters more than how much it's inefficient.
Test coverage is often collected with a profiler, or something very similar to a profiler. However, it’s not by itself concerned with performance.

You aren’t measuring your code when you collect coverage data, you’re measuring your tests. You’re trying to determine how well your tests exercise the code.

Quote:
Gotcha, so if a particular test fails you can get more accurate information about where in your code the bug lies.
No. Test coverage tells you what parts of your code are definitely untested.

Here’s a formal definition of test coverage I came up with a few years ago, if it helps.

Take some set of source code locations, L, which maps bijectively onto the lines of code in some collection of source code (e.g. each element is an ordered pair of source file name and line number). We say that line x in L is run (in R) if the line of code at location x is executed during any test. We say that line x in L is tested (in T) if any single test both executes the line and asserts the result of that execution (N.B. T is a subset of R.)

Test coverage partitions L into three disjoint subsets: covered, uncovered, and uncoverable. Covered is a subset of R. Uncovered is disjoint from R. No other information is known.

(The ‘filtering’ I was talking about ^ makes covered a subset of T, instead of a subset of R. A slightly better position.)

Quote:
I'm guessing GNU builds the lookup table. Without a kind of Perforce setup, does the extra time it takes to build the table significant? My intuition says probably not.
GCC builds the lookup table. It has to be regenerated any time the translation unit changes, but it can be cached, and indeed ccache does so.

Originally posted by Reid:
Yeah, I think I see that now. The tools are going to "slow down" development time in some sense, and I think my previous thoughts were latching more onto the increased amount of computation needed, but again that article made it clear in which ways the tools are cutting down lost time by effectively making it much easier and quick to spot and rollback code that's not working.
Some of them might slow you down initially, for sure. Code reviews definitely have a high perceived cost, for example. But so does testing. And so does writing a spec. And so does using an SCM, a 40 hour work week, and so-forth. You have to decide where to draw the line, which tools or processes are worth that up-front cost.

fwiw, though, roughly 100% of the software industry presently draws that line faaaaaaar short of the break-even point.

Quote:
No, I haven't. I have seen very little in theoretical computer science that's not related to geometric group theory, I'm afraid.


They are exceedingly mathy.
2018-01-03, 3:28 AM #19
Originally posted by Reid:
So like, if you're going to run your own company, set up a server farm with Perforce and develop a ton of tools to compete with Google? I mean, sounds nice, but aren't the barriers to achieve that sort of workflow high and wouldn't it take a long time to develop?
No, no, and no:

Quote:
Google doesn't have clean Makefiles or Antfiles, they have a build system with a well-specified, queryable, partition-tolerant dependency graph.
This is already available today. Bazel, Buck, Pants, and some from other lineages like build2 and Gradle. You don’t need to build this from scratch, you just need to not use Makefiles and MsBuild.

But by all means, if you want to do something interesting with your project graph, you should consider rolling your own build system eventually. It is not that labor intensive.

Quote:
Google doesn't give developers a powerful workstation, they've got a build system that can dispatch builds and tests to an entire server farm.
If you can afford enough developers to make a problematic number of tests, you really can’t afford not to throw money at this.

Quote:
Google doesn't fire off a build and test every time someone lands code, they do it before the code is allowed to land.
Phabricator + Jenkins

Quote:
They don't lint every few days, and they don't get coverage every few days, they do it for every commit, and that information is included in every code review.
Phabricator + Jenkins


Not sure where Perforce fits into this. Surely you’d use git, unless you have some specific reason not to. And at that point you’d probably use svn. If you can afford the full-time p4 admin it takes to keep it running, I guess you can do that if you really want.
2018-01-03, 4:36 PM #20
What a disaster. Not just Intel processors, and not just native code-even Javascript on the latest Chrome can exploit Spectre!

Code: 
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

 Xen Security Advisory CVE-2017-5753,CVE-2017-5715,CVE-2017-5754 / XSA-254
                              version 2

        Information leak via side effects of speculative execution

UPDATES IN VERSION 2
====================

Added CVEs.

ISSUE DESCRIPTION
=================

Processors give the illusion of a sequence of instructions executed
one-by-one.  However, in order to most efficiently use cpu resources,
modern superscalar processors actually begin executing many
instructions in parallel.  In cases where instructions depend on the
result of previous instructions or checks which have not yet
completed, execution happens based on guesses about what the outcome
will be.  If the guess is correct, execution has been sped up.  If the
guess is incorrect, partially-executed instructions are cancelled and
architectural state changes (to registers, memory, and so on)
reverted; but the whole process is no slower than if no guess had been
made at all.  This is sometimes called "speculative execution".

Unfortunately, although architectural state is rolled back, there are
other side effects, such as changes to TLB or cache state, which are
not rolled back.  These side effects can subsequently be detected by
an attacker to determine information about what happened during the
speculative execution phase.  If an attacker can cause speculative
execution to access sensitive memory areas, they may be able to infer
what that sensitive memory contained.

Furthermore, these guesses can often be 'poisoned', such that attacker
can cause logic to reliably 'guess' the way the attacker chooses.
This advisory discusses three ways to cause speculative execution to
access sensitive memory areas (named here according to the
discoverer's naming scheme):

SP1, "Bounds-check bypass": Poison the branch predictor, such that
operating system or hypervisor code is speculatively executed past
boundary and security checks.  This would allow an attacker to, for
instance, cause speculative code in the normal hypercall / emulation
path to execute with wild array indexes.

SP2, "Branch Target Injection": Poison the branch predictor.
Well-abstracted code often involves calling function pointers via
indirect branches; reading these function pointers may involve a
(slow) memory access, so the CPU attempts to guess where indirect
branches will lead.  Poisoning this enables an attacker to
speculatively branch to any code that exists in the hypervisor.

SP3, "Rogue Data Load": On some processors, certain pagetable
permission checks only happen when the instruction is retired;
effectively meaning that speculative execution is not subject to
pagetable permission checks.  On such processors, an attacker can
speculatively execute arbitrary code in userspace with, effectively,
the highest privilege level.

More information is available here:
  https://meltdownattack.com/
  https://spectreattack.com/

Additional Xen-specific background:

64-bit Xen hypervisors on systems with less than 5TiB of RAM map all
of physical RAM, so code speculatively executed in a hypervisor
context can read all of system RAM.

When running PV guests, the guest and the hypervisor share the address
space; guest kernels run in a lower privilege level, and Xen runs in
the highest privilege level.  (HVM and PVH guests run in a separate
address space to the hypervisor.)  However, only 64-bit PV guests can
generate addresses large enough to point to hypervisor memory.

IMPACT
======

Xen guests may be able to infer the contents of arbitrary host memory,
including memory assigned to other guests.

An attacker's choice of code to speculatively execute (and thus the
ease of extracting useful information) goes up with the numbers.  For
SP1, or SP2 on systems where SMEP (supervisor mode execute protection)
is enabled: an attacker is limited to windows of code after bound
checks of user-supplied indexes.  For SP2 without SMEP, or SP3, an
attacker can write arbitrary code to speculatively execute.

NOTE ON TIMING
==============

This vulnerability was originally scheduled to be made public on 9
January.  It was accelerated at the request of the discloser due to
one of the issues being made public.

VULNERABLE SYSTEMS
==================

Systems running all versions of Xen are affected.

For SP1 and SP2, both Intel and AMD are vulnerable.

For SP3, only Intel processors are vulnerable. Furthermore, only
64-bit PV guests can exploit SP3 against Xen.  PVH and 32-bit PV
guests cannot exploit SP3.

We believe that ARM is affected, but unfortunately due to the
accelerated schedule, we haven't been able to get concrete input from
ARM.  We are asking ARM and will publish more information when it is
available.

MITIGATION
==========

There is no mitigation for SP1 and SP2.

SP3 can be mitigated by running guests in HVM or PVH mode.

For guests with legacy PV kernels which cannot be run in HVM mode, we
have developed a "shim" hypervisor that allows PV guests to run in PVH
mode.  Unfortunately, due to the accelerated schedule, this is not yet
ready to release.  We expect to have it ready for 4.10, as well as PVH
backports to 4.9 and 4.8, available over the next few days.

RESOLUTION
==========

There is no available resolution for SP1 or SP3.

We are working on patches which mitigate SP2 but these are not
currently available.  Given that the vulnerabilities are now public,
these will be developed and published in public, initially via
xen-devel.

When we have useful information we will send an update.

NOTE ON LACK OF EMBARGO
=======================

The timetable and process were set by the discloser.

After the intensive initial response period for these vulnerabilities
is over, we will prepare and publish a full timeline, as we have done
in a handful of other cases of significant public interest where we
saw opportunities for process improvement.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBCAAGBQJaTVp4AAoJEIP+FMlX6CvZTcwH/2DpfLGwINA0C3V0zy6WcJAu
zxj7oqjorODWMIZbyR+gdSJHX82PKEJVgAdH/wtzb/GSdFJ+D3Q+zwZQSq1hxCZr
g9Otd+u6PyACsrQRK8mIoahYKUgPjTQdK2mzkKTO8SF9dQB5MSFht1vLdjXXGaWn
ifMfzNXgr3UCs5fOhQga/f2UdkbLal/qi0H2mxPyXCgalb6MGpMWEgMcmoAlFqnM
7aRmgYWrGaPKRHw4wwePWty+KEoryzPdF1vtURw8k/wdEDjzWYGZbhyBcHTd1BG7
or/J7mIsfs8SO7vua/6+msTfHnsmyWgZPweM4dzcO1AUEHDN0dYz6TOqaFwJuew=
=pwaX
-----END PGP SIGNATURE-----


http://seclists.org/oss-sec/2018/q1/3
2018-01-03, 4:38 PM #21
From the Specre paper:

Quote:
Attacks using JavaScript.
In addition to violating process isolation boundaries using native code, Spectre attacks can also be used to violate browser sandboxing, by mounting them via portable JavaScript code. We wrote a JavaScript program that successfully reads data from the address space of the browser process running it.
2018-01-03, 4:44 PM #22
So between Meltdown, Spectre, and the Equifax breach, pretty much anybody can read both my computer's memory and also take out a line of credit in my name.

How long before exploitation of these leaks mature into off-the-shelf instructions, to the point where randomly getting pwned becomes commonplace?

Are we as a society going to have to start to rely on a herd mentality, hoping we're the lucky ones that don't get picked off? What's stopping bad people from using this stuff to empty the bank accounts of people they don't like?
2018-01-04, 10:24 AM #23
Spectre bug mitigation is landing in compilers. The fix? Break speculative execution of indirect function calls.

http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20180101/513630.html

Indirect function calls are important for a bunch of things. They’re used for calling function pointers, dynamic dispatch (virtual functions/class inheritance/most object oriented programming), jump tables (optimizing large switch statements), and for calling dynamically linked functions (DLLs/SOs/vDSOs).

Fortunately, user code doesn’t need to enable this mitigation - mostly. However, kernel code does. Even if you didn’t write your kernel in C++, there are only so many ways to implement something like a file descriptor - large switches or function tables - so, pretty much guaranteed, your optimized kernel is using indirect function calls all over the place.

Overhead costs of this fix range between 2% to 50%, depending on how the original code was written.
2018-01-04, 11:04 AM #24
That's pretty dramatic. Will be interesting to see if the major OS's get slow as the releases roll out. Heck, even my phone will probably slow down.
2018-01-05, 2:01 AM #25
Nah, your phone is safe. I mean, what are the chances that it actually will get updated? :-/
Sorry for the lousy German
2018-01-05, 2:04 AM #26
My condolences to non-Nexus device owners out there.
2018-01-05, 2:12 AM #27
Originally posted by Impi:
Nah, your phone is safe. I mean, what are the chances that it actually will get updated? :-/


Apple has to make your old phone slower somehow.
2018-01-05, 2:29 AM #28
I’ll suggest a long-term solution for these kinds of problems: use separate computers, possibly integrated together, for performance-critical and security-critical computations. Like having a programmable Secure Enclave, some primitive single-dispatch non-pipelined CPU that takes exactly 1 cycle for every instruction. Then you update all software, so things like video games run on the fast computer, and things like your password manager run on the second.

Then you take the second computer and throw it right in the garbage.

Boom. Problem solved.
2018-01-05, 2:40 AM #29
Totally had me until the second paragraph. Should have known that such enthusiasm for harebrained technotrash would have been distinctly out of character for you. :P

In all seriousness, I see folks on HN pondering the possibility of just winging it and reaping the performance gains obtained by not patching, so long as they're reasonably sure the machine won't run any suspect code.
2018-01-05, 2:43 AM #30
For example, what about projects like Julia. They built a language around dynamic dispatch but for scientific computation. If the code runs in a cluster of my control, who cares about these side channel attacks.
2018-01-05, 3:05 AM #31
If Julia really is a “language built around dynamic dispatch”, I have good news. You can switch to C and your code will run 1000x faster even on a patched machine.
2018-01-05, 10:38 AM #32
"The S in Intel stands for security"
2018-01-05, 11:10 AM #33
The good news is
2018-01-05, 11:16 AM #34
Originally posted by Jon`C:
If Julia really is a “language built around dynamic dispatch”, I have good news. You can switch to C and your code will run 1000x faster even on a patched machine.


Turns out I that the Julia compiler actually optimized away the "dynamic" part of dispatch whenever it can anyway.

Quote:
Dynamic (runtime) dispatch only occurs when the compiler can’t figure out the types. For example, if you do map(g, Any[1,2,Int32(3),4]), where it is applying g to an array of Any and therefore doesn’t know the type of x until runtime, it will compile a generic version of g that dispatches at runtime to the correct version of f.

This kind of aggressive type specialization and type inference is the key strength of Julia. It allows you to write really generic code that works for any type, and doesn’t declare any types, and yet which is at the same time fast (because it is specialized when the code calling it is compiled).


https://discourse.julialang.org/t/dynamic-dispatch/6963/2

Since they compile with LLVM, I imagine they'll get the fix for free.
2018-01-06, 6:47 AM #35
I wonder how the Intel issue is going to affect performance of programs that do large CAD work and structural analysis at my workplace. :/
SnailIracing:n(500tpostshpereline)pants
-----------------------------@%
2018-01-06, 8:16 AM #36
Originally posted by ECHOMAN:
I wonder how the Intel issue is going to affect performance of programs that do large CAD work and structural analysis at my workplace. :/


Probably not much. These fixes affect the performance of your operating system, not your programs. If your software spends most of its time analyzing data in memory, it won’t be affected. If your software spends most of its time doing small file reads, it’s going to hurt a lot. Exactly how you’ll be affected depends on how your programs were written, but I’d expect them to be closer to the former than the latter.
2018-01-06, 9:00 AM #37
Epic reports: this change made their TCP game server CPU utilization increase by roughly 100%.
2018-01-10, 6:20 AM #38
The rest of this week: all day at JMM with breaks to drive to meetings for work. And some social stuff, and a programming project to tie up. Then a twelve hour plane and train journey back home to Virginia, then I have to prepare for teaching and classes.

Then I'll have the free mental space to read the thoughtful replies carefully. Hurray for being torturously busy!
2018-01-10, 8:54 AM #39
i was complaining about machine learning yesterday.

my adviser wants me to pick up ML (specifically reinforcement learning for right now) for application to some physics problems. I've read the important chapters of Richard Sutton's book, and I think it was great, but I finished the chapters and just remember thinking "that's it?" Algorithms presented were dead simple (iterate and update, just change when you update!), the framework is... also very simple. There was so much jargon built on top of all of this that it struck me as nigh impenetrable, but this is math I could've tackled as a freshman.

Now the difficulty is picking up neural network material, which seems even worse about bull**** jargon. Every intro I read always hyped up the backpropagation algorithm to be this world-changing discovery that, surprise, it's just the chain rule written out explicitly. I haven't found a nice tutorial to actually pick up and learn this material because everything seems to be "babby's first neural net," and I'm unsure how much theory is really important to throw into the mix when I'm learning all of this.

I've rarely met a CS student who struck me as good at mathematics, which makes me feel all kinds of uneasy about work that they do. Is this field really just 90% jargon and knowing libraries?
I had a blog. It sucked.
2018-01-10, 9:08 AM #40
A really fancy Markov chain
1234567891011121314151617181920212223242526272829303132333435

↑ Up to the top!