Index

Show enters and exits. Hide enters and exits.

00:00:09brixennice
00:00:32evani don't know if I should go "yay fast blocks" or "boo slow allocation"
00:00:52goyox86last time i've followed ingalls he was working for SUN at research, on something called the lively kernel
00:01:08brixenhmm, well yay fast blocks since the jit is kicking ass basically
00:01:28evangoyox86_: he probably got the ax from oracle.
00:01:34evana lot of fellows did.
00:01:49goyox86evan: agree
00:01:54brixenif oracle axed them, they are stupid as hell
00:02:07brixenif they quit, woot for them
00:02:58brixenjudging from the little tim wrote, I'm guessing oracle is stupid as hell
00:03:05goyox86evan: probably oracle guys,didn't read "Design Principles Behind Smalltalk" lol
00:03:16evanhell
00:03:21evanyou just have to listen to dan's stories
00:03:26evanhe's worth every penny just for the fucking stories.
00:03:31brixenhehe
00:05:54postmodernoracle fired someone?
00:05:59goyox86that lively kernel project, was really nice, dan bringing a lot of his VM knowledge, to web browsers, as he stated "Web programming was more complex that it should be"
00:06:33goyox86than it should be*
00:07:10goyox86the guy implemented that stuff if JS
00:09:48brixengoyox86_: are you following the vpri work?
00:10:08evanwoop!
00:10:13slavaevan: I got GC maps working
00:10:23goyox86brixen: nope what is this about?
00:10:41brixengoyox86_: vpri.org
00:10:42evanbrixen: http://gist.github.com/450772
00:10:46evanslava: oh rad.
00:11:15slavaevan: each code block now has a list of return addresses and a bitmap associating each return address with a set of spill slots in the stack frame that are GC roots
00:11:38evanah rad.
00:11:40slavabefore I couldn't spill tagged values across subroutine calls and had to shuffle them onto the operand stack instead
00:11:55slavathis is going to make a lot of things much cleaner and more efficient
00:12:06evanbrixen: each_key_block == 1.43s with block inlining
00:12:11evaneach_key_to_iter == 3.2s
00:12:16slavathe operand stack will only be used for parameter passing, and not temporary value storage
00:12:18evaneach_key_direct == 1.40s
00:12:28goyox86brixen: kay and his folks nice!
00:12:41evanslava: very cool.
00:13:00evanslava: so, all GC pointers are always spilled
00:13:09evanand the GC uses the map to fix up the stack positions
00:13:21slavaeverything is spilled at a call site
00:13:24evanand post GC point, they're unspilled
00:13:28slavaI don't have non-volatile registers
00:13:29slavaright
00:13:32evangotcha
00:13:33evansweet.
00:13:34slavaso in the IR, its one SSA value
00:13:48evanaaah
00:13:59evanyou're doing it all at the lowest layer
00:13:59evannice!
00:14:04slavayeah, in the register allocator
00:15:03evanslava: say you have a GC ref in SSA variable $a
00:15:04evanand you do
00:15:32evan$b = position_of_class($a)
00:15:39evancall(...)
00:15:43evan$c = load($b)
00:16:11evanor does your SSA not have variables that would reflect on the memory of another variable
00:16:13evanLLVM does that
00:16:34slavaok, so the way things work right now, I don't create pointers into the middle of objects like that
00:16:45evangotcha
00:16:46slavabut eventually, this will be supported
00:16:54evanthats a fundemental way LLVM works
00:17:00evanwhich is why i'm worried about this kind of stack mapping
00:17:16slavausually, you only support a small subset of all possible pointer arithmetic expressions
00:17:35evanLLVM only really supports one
00:17:41slavafor example, if $a and $b are GC pointers, and you do
00:17:43slava$c = $a - $b
00:17:45slavacall(...)
00:17:46slavablah($c)
00:17:50slavathen you just suck.
00:17:59evanwhich is the ability to ask for a pointer to a struct member
00:18:01slavathere's no meaningful way to fix up $c
00:18:04evanthat you can then load
00:18:14slavabut this is easy to handle:
00:18:19slava$c = $a + 0x123
00:18:20slavacall(...)
00:18:22slavablah($c)
00:18:48slavawhat you do is record that $c's base pointer is $a
00:18:58slavathen the GC computes $c - $a before GC
00:19:06slavaand sets $c to $a plus the saved difference after GC
00:19:56slavaso in addition to a set of spill slots which are GC roots, you also have a hashtable mapping spill slots to base pointers, and treat those by saving the differences and adding them on again after
00:21:02slavaevan: the hard case is if you have
00:21:05slava$a = $x + 123
00:21:06slava...
00:21:09slava$b = $x + 321
00:21:10slava...
00:21:13slava$c = phi($a,$b)
00:21:20slavacall(...)
00:21:21evanyeah
00:21:51slavaI don't understand the trick for this to be honest, but I know hotspot server does it properly
00:22:06slavathe paper I'm reading that describes GC maps is about Modula 3 and their compiler was not SSA.
00:22:54evanwe should get to the bottom of that.
00:23:00evani'd like to know.
00:23:09slavaare you thinking of going with a similar approach?
00:23:11evanbecause eventually i want to work on this in the LLV codebase.
00:23:16evani'd like to
00:24:15evanbecause it properly express something I can't express now
00:24:55evannamely: This thing might have moved, but all other knowledge you had about it is still valid
00:25:27slavayeah, since in our VMs, 'knowledge' does not include 'the 5th bit of the pointer value is 1'
00:27:24evanyep.
00:27:35evanstuff like "the class was a String"
00:27:39evan"it's got 15 fields"
00:27:40evanetc.
00:27:52evanthe class was String, rather.
00:28:22slavaso far I'm only using GC maps at FFI call sites
00:28:32brixenslava: hmm, that seems like: x + y == f(x) + f(y) where f is the delta applied to addr(x) and addr(y)
00:29:06brixenok, that's not exactly right
00:29:22brixenI can see this in functions though, hrm
00:30:21evanbrixen: see my gist?
00:30:26brixenf (GC) is a structure preserving transform
00:30:31brixenevan: yeah
00:30:32evanusing each_entry{} is faster than #to_iter
00:30:36brixenyep
00:30:44brixenbecause to_iter has an allocation cost
00:30:44evanyaypants!
00:31:03brixenthere's no way to beat a pure inlined loop :)
00:31:12evanwell, interestingly enough
00:31:15evanso does each_entry {}
00:31:25evanbecause it's got the block allocation without block inlining
00:31:34evanand without block inlining, it's still faster.
00:31:39evanwhich is weird.
00:31:48brixenI don't see each_entry doing a n allocation
00:31:57evann?
00:32:02evanwhen you call it
00:32:06evanyou pass a block
00:32:11evanpassing a block in an allocation
00:32:24evanit's lowlevel, but it's an allocation.
00:32:36brixenahh, and this is faster even without inlining?
00:32:39brixeninteresting!
00:33:31brixenI'd like to know why that is
00:33:48evanas would I
00:33:51evani'm going to poke a bit.
00:33:55brixenok
00:34:19evanside note:
00:34:39evanin the back of my mind, i'm still wondering about the ability to have "constant" blocks
00:34:44evanlike in each_key_block
00:34:51brixenyeah
00:34:51evanthe block doesn't capture any locals.
00:35:21brixenseems like we should be able to analyze that
00:36:33brixenlike a block has an attribute about the outermost scope it actually closes over
00:37:41evanbrixen: elsewhere i'm discussing http://coderoom.wordpress.com/2010/06/23/criminal-overengineering/
00:37:58evanand i came up with a good way to sum things up, code complexity wise.
00:38:04evansimple is harder than complex
00:38:11evananyone can bang on keyboard and output 1000 pages
00:38:20evanit takes skill to sum up the human condition in a 20 word poem.
00:39:27brixenindeed
00:39:39evanin science, I believe they say that if you can't explain something simply to a lay person, you don't understand it well enough.
00:39:56brixento quote Einstein, yes :)
00:40:02evan:)
00:40:08brixenalso interesting in that post is the reference to fear
00:40:18brixenpeople are inordinately motivated by fear
00:40:38brixenor, motivated disproportionately by fear
00:40:38evani think that partly comes from managers not know how to judge the output of a programmer
00:40:49evaneveryone has heard of being paid per line of code
00:40:51evaneven if they're not.
00:40:58brixenie, they do not have the ability to distinguish how much their fear should motivate them
00:41:27brixenyeah, but also the "oh god, what if Bob thinks this code sucks"
00:41:34brixeninsert your own Bob
00:41:56evanyep.
00:42:22evanthats why i'm motivated to show praise on obvious solutions
00:42:27evans/show/shower/
00:42:32evanobvious solutions are awesome
00:42:33brixenyeah
00:42:44evanthey're obvious because they're simple.
00:42:54brixenI also think that moving the fear burden to tests helps
00:43:01evantotally.
00:43:11brixenyou can code simple because the tests help you control your irrational fears
00:43:13evanit helps quell the "but what if!"
00:43:22brixentests incomplete? no problem, write a new case
00:43:35evanin fact, the same is true of managed languages
00:43:52evanmissed cases doesn't mean crashing and coming in on a sunday
00:44:45evanbecause that is stress
00:44:51brixenyeah
00:45:03evanand humans feel stress as fear and fear as stress.
00:45:58evanwe are but mearly clinging to sanity in a sea of emotions anyway
00:46:12brixenI think the most productive question is often "what's the worst that can happen?"
00:46:38brixenone has to realistically evaluate that
00:46:43brixenrepeatedly
00:46:49evanand be honest with the answer
00:46:54brixenyeah
00:47:42evanbrixen: so, i'm looking at the difference between each_entry and to_iter
00:47:45evanwith -Xprofile, so no JIT
00:47:55evani'll show ya the profile output
00:47:58evani have a hunch.
00:48:04brixenok
00:48:46evanhttp://gist.github.com/450809
00:48:53evancheck the file names for which is which
00:48:58evanthough you can see by the methods called
00:49:11evan3s difference in runtime when profiling
00:49:24evanwhich can easily be accounted to the calles to #allocate and #next
00:50:32evanbut when running without profiling
00:50:47evaneach_entry == 1.8s, to_iter == 3.9s
00:51:08evanso my hunch is that the shape of each_entry is much easier to JIT
00:51:35evanit's a self call, and keeps the index and entries in locals
00:51:37brixenhmm, there are 1M more calls in the #to_iter case
00:51:52evanyes
00:51:55brixenyeah, that makes sense
00:52:00evanbecause you have #next as the condition as well
00:52:06brixenlocals are screaming fast
00:52:09evanso there is a final call to find out there are no more
00:52:11brixenwhen I benched them
00:52:25brixenyeah, that extra call is a lot
00:53:09brixenit will be an interesting project to make these equivalent in the jit
00:53:46slavathat's basically how I developed my compiler, figuring out how to make various high level idioms compile faster
00:53:53brixenslava: do you know what I mean by a structure preserving transform (in graphics terms)
00:54:05slavamore generally that's called an isomorphism
00:54:13brixenslava: seems like the jit should see GC as a f() of this form
00:54:20brixenyes, an isomorphism
00:56:06brixenslava: basically, f() is pure
00:56:24slavaI think there's a way to tell LLVM that a function call won't mutate any memory cells
00:56:32slavaand alias analysis will have more precise information around this call as a result
00:56:45evanthere is
00:56:47evanand i've done that
00:56:53evanalso you can write your own AA pass
00:57:01slavadoes it help if you mark the actual GC call in this manner?
00:57:01evanto provide that info using whatever knowledge you want.
00:57:14evanit crashes the program.
00:57:19brixen:(
00:57:21slavaI'll take that as a "no" :)
00:57:31evanbecause LLVM doesn't reload things it must
00:57:41evanbecause you've told it "nothing you can see right now will change"
00:57:44evaneven though it does change.
00:57:57brixenyes, addr(x) != f(addr(x))
00:58:07brixenbut every thing else is the same
00:58:12brixen-\s
00:58:13evanright
00:58:18evanso my plan is to build things on top of that
00:58:20slavaright, the addresess might change, you just can't observe the difference without breaking abstractions
00:58:21evanin 2.7
00:58:25evanusing metadata and our own AA
00:58:35brixenahh cool
00:58:44evanwhich is, imho, hte right way to go away
00:58:51evanbecause that gives us all the flexibility we want.
00:59:12evanLLVM becomes our lowlevel IR (OH WAIT, ISN'T THAT THE NAME?)
00:59:15evan:D
00:59:18brixenhehe
00:59:30slavamy high level IR can 'see' values across calls
00:59:31brixenthose jokesters
00:59:35slavabut in LLIR the operand stack ops become explicit
00:59:45slavaso a constant defined before a call is not known to be a constant after the call in the later stages
00:59:47evanbrixen: i know! it's almost like Chris got the ACM award for a reason!
00:59:53brixenheh
01:00:02slavaby moving the GC stuff to register allocation, more passes will 'see' across calls
01:00:04evanslava: exactly, i've thought about having a high level IR for a while now
01:00:09evanthat can do exactly this.
01:00:30evansee things and not worry about address
01:00:50evanthe high level concept of a reference
01:00:59evanrather than the lowlevel concept of a pointer.
01:01:33brixenseems like metadata gives you that
01:01:47evanit does
01:01:49slavathe two IRs are a bit of a historical accident though
01:01:58slavaLLIR represents control flow with a graph of basic blocks
01:01:58evanthe metadata is basically another layer of info
01:02:07slavaHLIR is more like an AST, it has an #if node with two children
01:02:15evanthat we can propagate based on high level reference concepts
01:02:25evanslava: aah.
01:02:32evanyour HLIR is like our bytecode.
01:02:40slavayes
01:03:07slavaand optimization passes that operate on it primarily do inlining or replacing subroutine calls to known words with cheaper operations
01:03:16evancheating!
01:03:21evan:D
01:03:31slavaideally everything would operate on one IR
01:03:39slavabut it would mean rewriting a ton of code
01:03:57brixenslava: when are you writing the book on dynlang vms? :)
01:04:06slavaand there wouldn't be a practical benefit, it would just make the code more elegant
01:05:16evanslava: I suspect you'll do it eventually.
01:05:18evan:)
01:05:30slavayeah :)
01:11:59evanarg.
01:12:03evanJIT bug
01:12:06evanit JIT'd a script body
01:12:09evanoops.
01:12:17brixenock
01:12:22evan</waste of time>
01:19:15evanrad, the mexicali quake over easter moved the crust by 31 inches
01:19:58evan10 feet in some places!
01:22:29brixenseems like a lot, but I wonder what that is by % of size of the plate
01:23:03brixenprobably less than me moving this 4ft table < 1mm
01:23:12brixenoops mixing units there :)
01:26:40evanhehe
01:27:07jakedouglashi rubinius
01:27:30brixenhey jakedouglas
01:27:37jakedouglasbeen a while :p
01:27:42brixenheh
01:27:48brixendo you bring us good tidings?
01:28:27jakedouglasjust congrats on 1.0 and beyond :)
01:28:42brixensweet, thanks! :)
01:29:18jakedouglasmy conscience is still haunted occasionally by a spec i said i would write but never did
01:29:23jakedouglasi might actually do it one of these days
01:29:30brixenhah
01:29:57brixenyou should, lest your troubled soul haunts you indefinitely
01:29:58brixen:)
01:30:01jakedouglasi remember fixing the bug but the spec looked like a real bitch so i told myself i would do it later..
01:30:26jakedouglasyea, i know :)
01:32:26brixenI'm going to grab dinner, lest my stomach growls at me indefinitely :)
01:33:25jakedouglasheh
02:08:52bakkdoorhello
05:32:48dbussinkevan: i went to bed yeah
05:32:57evan:D
05:34:45dbussinkevan: did see you found some surprising benchmark results :)
05:34:52evanyeah
05:35:09evansome 2x speed ups in Hash iteration
05:35:24evanworking on speeding it up more now.
05:35:32evangot an idea while running today
05:35:34dbussinkevan: i wonder, would the 0 entries be worth optimizing by returning directly?
05:35:39evani've fixed up some old profiling code I did for the jit
05:35:57evanand made it so that -Xjit.profile turns on this profiling code in the JIT
05:36:06evanso that with that option, methods can show up twice in the profiling
05:36:08brixendbussink: evan is making deals with the devil, at your prompting
05:36:11evanHash#each_entry
05:36:12evanand
05:36:14evanHash#each_entry <jit>
05:36:18evan2 seperate entries
05:36:23dbussinkah ok, cool :)
05:36:26evanso you can see exactly what the JIT caused.
05:36:31brixenthat's nice
05:36:34dbussinkso you could see difference there too
05:36:40evanso that when it says "oh, I inlined that thing"
05:36:48evani can actually check if it used the inline version
05:36:55evanbeacuse if so, then that method disappears from the profiling
05:37:07evanbecause no inline method contributes profiling info.
05:38:06evanthats confusing to a normal user
05:38:09evanbut it tells me a lot.
05:39:14evancheck it
05:39:14evanhttp://gist.github.com/451031
05:39:33evannote the 2 entries for Hash#each_key
05:39:44evanone jit, one not.
05:40:16evanadditionally, Hash#each_key_block {} count stopped increasing
05:40:26evanthough, i don't have block inlining on
05:40:29evanneed to look into that.
05:40:33brixenheh
05:41:02evanwowzers.
05:41:04evancheck this shit.
05:41:17evanwait wait
05:41:23evani'm missing some data..
05:42:20brixenI thought block inlining was enabled by default again?
05:43:09evandid I?
05:43:14evanno not yet
05:43:53evanreload the gist
05:44:45evani'm missing some block data
05:44:46evanI think.
05:45:18evanbecause my guess is that everything got inlined into the block passed to #times
05:45:23evanbut that row is missing.
05:47:05brixenevan: could you gist hash_create.rb
05:47:44evanreload
05:47:47evanit's not creating hashing anymore
05:47:51evanbut i didn't change the file name
05:48:12evanah, I see why.
05:49:05evani've wired and rewired block invocation so many times
05:49:07evani confused myself!
06:23:29evanrab
06:23:31evanrad
06:23:32evanfixed it.
06:24:13evanbrixen: http://gist.github.com/451031
06:24:18evancheck out the 2nd file there
06:24:27evanthe jit'd block now shows up
06:24:46evanand we can see that pretty much all the code got inlined into it
06:25:00brixenyeah
06:26:44evanthats good knowledge.
06:28:07brixenblock inlining in this case certainly is a win
06:28:15evanyeah
06:28:31evanthough, i think there is still a bug in block inlining here
06:28:35evanit's inlining more than I expect it to.
06:28:48evani thought I made block inlining more restrictive
06:28:55evanmaybe this is actually ok..
07:08:03dbussinkevan: is it inline too much so that it hurts performance?
07:08:16evani'm headed to bed
07:08:18evanbut real fast
07:08:24evanyes, but thats not the case i'm worried about here
07:08:28evanblock inlining is quite tricky
07:08:33evanbecause the control flow is very strange.
07:08:54evanso i'm worried it inlined too much and would make the control flow weird
07:08:58evani'll debug it more tomorro
07:08:59evanw
07:09:05evani'm off to bed
07:09:08dbussinkevan: ah, sleep well :0
07:09:09dbussink:0
07:09:11dbussink:)
07:09:13dbussinki can't type :S
14:37:20manveruevan: around?
19:58:53spastorinoI've just run the Rails tests on Rubinius http://gist.github.com/451865 impressive results guys
19:59:45Defilercool, yeah, all those failures except memcache look like bad tests
19:59:51Defilere.g. ones that rely on the order of keys in a Hash
20:00:00Defileror ones that assert things about platform-specific date formats
20:00:43Defilerhehe, you should see if PPC Process Gearbox works on rbx
20:00:50Defilerassuming you guys still have the code
20:01:52spastorinoyeah i have seen this tests i will fix them ;)
20:14:31eventualbuddhaI'm getting the same error message described at http://gist.github.com/405422: "Could not find RubyGem rake" while installing rbx with rvm on osx. anyone know what that's about?
20:15:35Defilerare you using rvm head?
20:15:40dbussinkeventualbuddha: that's a pretty old rvm
20:15:48Defilernot his output
20:15:49dbussinkeventualbuddha: i suggest updating first :)
20:15:50Defilerhe's linking to an old gist
20:15:59dbussinkah ok, sorry
20:16:04Defilerrvm update --head
20:16:07Defilerand see if you get the same error
20:16:27eventualbuddhadbussink: well, that wasn't my gist
20:16:29Defilerif you do, you may want to ask in #rvm, though wayne definitely hangs out here too
20:16:31eventualbuddhabut I'll try updating
20:21:34eventualbuddhaDefiler, dbussink: https://gist.github.com/d8d8bcc37259a4d812be
20:21:39eventualbuddhamy actual output
20:22:27jaribi'm still having trouble building on arch linux with 1.9; anyone know how to solve this? http://gist.github.com/451926
20:26:52cyndisjarib: try removing lib/ext/openssl/Makefile and then try again
20:27:01jaribok
20:37:41Defilereventualbuddha: ok.. do "rvm implode" and then install it from head
20:37:53Defilere.g. bash < <( curl http://rvm.beginrescueend.com/releases/rvm-install-head )
20:48:08eventualbuddhaDefiler: thanks. imploded, reinstalled, did `rvm install rbx` and got this: https://gist.github.com/fbd909bb7ac1faa8e3c6
20:53:08Defilereventualbuddha: rbx-copy wtf is that
20:53:17Defilereventualbuddha: sorry, never ever seen that.. I recommend asking in #rvm :(
21:01:33eventualbuddhaDefiler: thanks for the help
21:01:36eventualbuddhaI am ;)
21:01:41Defilercool
21:01:48Defilerlet me know what fixes it if it gets fixed
21:01:55eventualbuddhawill do
22:04:14bougymanoi
22:05:11bougymani'm getting a 404 on the git clone
22:05:21bougymanyou have an alternate repo?
22:05:44bougymannm, i got it