Hey Wonks:
I am currently playing with some concurrency and parallelism models. Does
anyone have some good pointers on Concurrent ML?
Aaron W. Hsu
--
Programming is just another word for the lost art of thinking.
_______________________________________________
___________________________________________________
Posted on the PL-wonks mailing list. Go to http://www.cs.indiana.edu/mailman/listinfo/pl-wonks to subscribe.
Jocaml has an interesting model.
On Feb 5, 2011, at 8:18 PM, "Aaron W. Hsu" <> wrote:
> Hey Wonks:
>
> I am currently playing with some concurrency and parallelism models. Does
> anyone have some good pointers on Concurrent ML?
>
> Aaron W. Hsu
>
> --
> Programming is just another word for the lost art of thinking.
> _______________________________________________
___________________________________________________
Posted on the PL-wonks mailing list. Go to http://www.cs.indiana.edu/mailman/listinfo/pl-wonks to subscribe.
> On Feb 5, 2011, at 8:18 PM, "Aaron W. Hsu" <> wrote:
>
> Hey Wonks:
>
> I am currently playing with some concurrency and parallelism models. Does
> anyone have some good pointers on Concurrent ML?
Well, you can do CML in Haskell: http://hackage.haskell.org/package/cml
And there's the obvious: http://cml.cs.uchicago.edu/
If you're considering the CSP model, then there's some Haskell for
that too: http://www.cs.kent.ac.uk/projects/ofa/chp/
--
Live well,
~wren
_______________________________________________
___________________________________________________
Posted on the PL-wonks mailing list. Go to http://www.cs.indiana.edu/mailman/listinfo/pl-wonks to subscribe.
Hey Wren:
On Sun, 06 Feb 2011 00:26:42 -0500, wren ng thornton
<> wrote:
>> On Feb 5, 2011, at 8:18 PM, "Aaron W. Hsu" <> wrote:
>>
>> Hey Wonks:
>>
>> I am currently playing with some concurrency and parallelism models.
>> Does
>> anyone have some good pointers on Concurrent ML?
>
> Well, you can do CML in Haskell: http://hackage.haskell.org/package/cml
>
> And there's the obvious: http://cml.cs.uchicago.edu/
>
> If you're considering the CSP model, then there's some Haskell for
> that too: http://www.cs.kent.ac.uk/projects/ofa/chp/
Thanks for the pointers. It seems that both CSP and CML are examples of
synchronous message-passing concurrency. I am also reading about some
asynchronous models, of which Erlang, apparently, is one. Is there any
place that surveys these models (and others) fairly well?
Aaron W. Hsu
--
Programming is just another word for the lost art of thinking.
_______________________________________________
___________________________________________________
Posted on the PL-wonks mailing list. Go to http://www.cs.indiana.edu/mailman/listinfo/pl-wonks to subscribe.
On Mon, Feb 7, 2011 at 2:19 PM, Aaron W. Hsu <> wrote:
> Hey Wren:
>
> On Sun, 06 Feb 2011 00:26:42 -0500, wren ng thornton
> <> wrote:
>
>>> On Feb 5, 2011, at 8:18 PM, "Aaron W. Hsu" <> wrote:
>>>
>>> Hey Wonks:
>>>
>>> I am currently playing with some concurrency and parallelism models.
>>> Does
>>> anyone have some good pointers on Concurrent ML?
>>
>> Well, you can do CML in Haskell: http://hackage.haskell.org/package/cml
>>
>> And there's the obvious: http://cml.cs.uchicago.edu/
>>
>> If you're considering the CSP model, then there's some Haskell for
>> that too: http://www.cs.kent.ac.uk/projects/ofa/chp/
>
> Thanks for the pointers. It seems that both CSP and CML are examples of
> synchronous message-passing concurrency. I am also reading about some
> asynchronous models, of which Erlang, apparently, is one. Is there any
> place that surveys these models (and others) fairly well?
I can't think of any good surveys off-hand. Most of the ones I found
before were from the previous parallelism boom, before people
abandoned parallelism for faster Pentiums; and I haven't gone looking
for surveys since the new parallelism boom started.
I don't remember the specific details of Erlang's
parallelism/concurrency model, but how asynchronous does it need to
be? You may want to look at some of the papers about Haskell's STM,
it's on the asynchronous lock-free end of things and draws heavily
from work in the database community. Depending on your goal, it may be
worth contrasting Haskell's STM (which is awesome) with F#'s STM
(which is apparently too awful to be worth using[1])--- the difference
stems mainly from other language features, e.g. whether impurities
like mutability etc can be cordoned off by the type system.
What exactly are you looking for? And are you really looking for
parallelism, or concurrency, or both? SIMD, or SMP? normal computers,
or supercomputers?
Some other projects you may be interested in looking at:
Eden : http://www.mathematik.uni-marburg.de/~eden/
Moby : http://moby.cs.uchicago.edu/
Polyphonic C# : http://research.microsoft.com/en-us/um/people/nick/polyphony/
C_omega : http://research.microsoft.com/en-us/um/cambridge/projects/comega/
Orc : http://orc.csres.utexas.edu/
And of course the classics: CSP, pi-calculus, StarLisp / C^star, petri
nets, MPI,...
[1] I forget whether it was MSR or Intel who did a study of converting
some of their code to use STM and benchmarked the results, coming to
that conclusion. Probably MSR, but my google-fu is failing me at the
moment.
--
Live well,
~wren
_______________________________________________
___________________________________________________
Posted on the PL-wonks mailing list. Go to http://www.cs.indiana.edu/mailman/listinfo/pl-wonks to subscribe.
On Mon, 07 Feb 2011 20:55:27 -0500, wren ng thornton
<> wrote:
> What exactly are you looking for? And are you really looking for
> parallelism, or concurrency, or both? SIMD, or SMP? normal computers,
> or supercomputers?
I have heard much and read little about parallelism and concurrency (I am
interested in both concepts). Unfortunately, what I have heard is too
general to be practical, and what I have read is too theoretical and
abstract to be practical. The only real anecdotes I have of people doing
serious parallelism seems to be centered around the C++/MPI crowds. I have
difficulty fathoming that there are not other "real" users of the myriad
of concepts which appear to float around academic paper space.
In other words, say I am just a simple programmer who cares about
systematically adding parallelism and/or concurrency to his toolbox, what
does such a tool look like? I am trying to understand what these concepts
look like in real world programs and what sort of interfaces are both
efficient and pleasant to use. What sort of standard practices surround
the use of said interfaces and what collective wisdom exists on its use;
how does one use concurrency/parallelism in a general, responsible manner?
I cannot imagine using either MPI or the Pi calculus directly in my
programs, and in systems like Fortress, I struggle to extract the core
elements of parallelism design from the rest of the languages design
choices.
I hope that makes sense.
Aaron W. Hsu
--
Programming is just another word for the lost art of thinking.
_______________________________________________
___________________________________________________
Posted on the PL-wonks mailing list. Go to http://www.cs.indiana.edu/mailman/listinfo/pl-wonks to subscribe.
Aaron,
Those are all excellent questions and concerns that all of us are struggling to understand :).
You have probably already heard from Will and Eric about our so-called "declarative parallel programming" project, which is an attempt at addressing some of your concerns. However, I doubt you will find a practically usable tool today that satisfies all your requirements. In particular, "efficient" and "pleasant to use" might be difficult (impossible?) to come by in a single system. I am afraid, effective parallel programming is still a largely open problem.
Best,
Arun
On Feb 11, 2011, at 12:12 AM, Aaron W. Hsu wrote:
> On Mon, 07 Feb 2011 20:55:27 -0500, wren ng thornton
> <> wrote:
>
>> What exactly are you looking for? And are you really looking for
>> parallelism, or concurrency, or both? SIMD, or SMP? normal computers,
>> or supercomputers?
>
> I have heard much and read little about parallelism and concurrency (I am
> interested in both concepts). Unfortunately, what I have heard is too
> general to be practical, and what I have read is too theoretical and
> abstract to be practical. The only real anecdotes I have of people doing
> serious parallelism seems to be centered around the C++/MPI crowds. I have
> difficulty fathoming that there are not other "real" users of the myriad
> of concepts which appear to float around academic paper space.
>
> In other words, say I am just a simple programmer who cares about
> systematically adding parallelism and/or concurrency to his toolbox, what
> does such a tool look like? I am trying to understand what these concepts
> look like in real world programs and what sort of interfaces are both
> efficient and pleasant to use. What sort of standard practices surround
> the use of said interfaces and what collective wisdom exists on its use;
> how does one use concurrency/parallelism in a general, responsible manner?
> I cannot imagine using either MPI or the Pi calculus directly in my
> programs, and in systems like Fortress, I struggle to extract the core
> elements of parallelism design from the rest of the languages design
> choices.
>
> I hope that makes sense.
>
> Aaron W. Hsu
>
> --
> Programming is just another word for the lost art of thinking.
> _______________________________________________
___________________________________________________
Posted on the PL-wonks mailing list. Go to http://www.cs.indiana.edu/mailman/listinfo/pl-wonks to subscribe.
On Fri, 11 Feb 2011, Aaron W. Hsu wrote:
> On Mon, 07 Feb 2011 20:55:27 -0500, wren ng thornton
> <> wrote:
>
>> What exactly are you looking for? And are you really looking for
>> parallelism, or concurrency, or both? SIMD, or SMP? normal computers,
>> or supercomputers?
>
> I have heard much and read little about parallelism and concurrency (I am
> interested in both concepts). Unfortunately, what I have heard is too
> general to be practical, and what I have read is too theoretical and
> abstract to be practical. The only real anecdotes I have of people doing
> serious parallelism seems to be centered around the C++/MPI crowds. I have
> difficulty fathoming that there are not other "real" users of the myriad
> of concepts which appear to float around academic paper space.
>
> In other words, say I am just a simple programmer who cares about
> systematically adding parallelism and/or concurrency to his toolbox, what
> does such a tool look like? I am trying to understand what these concepts
> look like in real world programs and what sort of interfaces are both
> efficient and pleasant to use. What sort of standard practices surround
> the use of said interfaces and what collective wisdom exists on its use;
> how does one use concurrency/parallelism in a general, responsible manner?
> I cannot imagine using either MPI or the Pi calculus directly in my
> programs, and in systems like Fortress, I struggle to extract the core
> elements of parallelism design from the rest of the languages design
> choices.
The "serious" parallelism that you hear about around IU using MPI is from
the people using really large machines that need that kind of low-level
communication control. On multicore systems, people are more likely to
use things like threads for I/O overlap, task parallelism
(Cilk/TBB/PFunc), OpenMP, and MapReduce/Hadoop, all of which are much
easier to program than MPI and would fit in better with abstract models of
parallelism. I'm unfortunately too much in the academic world to know too
well what people really do, but the things I listed should be closer to
practice.
-- Jeremiah Willcock
_______________________________________________
___________________________________________________
Posted on the PL-wonks mailing list. Go to http://www.cs.indiana.edu/mailman/listinfo/pl-wonks to subscribe.
A lot of work was done at IU in the 70s and 80s on concurrent/parallel functional languages.
A novel, demand-driven, data-centric operational model was used. My language work
centered on stream-networks and speculative parallelism. I'd be happy to give an overview
is anyone is interested.
On 2/11/11 12:49 AM, Jeremiah Willcock wrote:
> On Fri, 11 Feb 2011, Aaron W. Hsu wrote:
>
>> On Mon, 07 Feb 2011 20:55:27 -0500, wren ng thornton
>> <> wrote:
>>
>>> What exactly are you looking for? And are you really looking for
>>> parallelism, or concurrency, or both? SIMD, or SMP? normal computers,
>>> or supercomputers?
--
Steve Johnson
www.cs.indiana.edu/~sjohnson
_______________________________________________
___________________________________________________
Posted on the PL-wonks mailing list. Go to http://www.cs.indiana.edu/mailman/listinfo/pl-wonks to subscribe.
Aaron, I think the answer depends a lot on the kind of problem you are
trying to solve, as the different parallel programming tools each seem to be
targeted at particular classes of applications. As others have mentioned, it
also depends on the hardware you're targeting, as different algorithms work
better on different types of hardware (for example, see EigenCFA, where they
converted CFA into a linear algebra problem that was well-suited for GPUs).
If your problem is a traditional scientific computing problem, MPI's not
actually all that bad. You might also look into some of the PGAS languages
like UPC, which make parallel programs look a little more like the programs
we're used to.
For less regular problems, like graph algorithms, something like active
messages or task parallelism is probably better.
For something like a web server, the Erlang or Scala models seem like they
would work pretty well.
In my own programs, a lot of times I end up making threadpools and a queue
of work items. This tends to be pretty lightweight (from a programmer effort
standpoint), and can be built into most languages without too much code as
long as you've got nice thread primitives. The downside is it doesn't
actually perform all that well.
You might look into promises and futures. They seem to be a way to say "I'll
need the result of this computation sometime, so go ahead and compute it
when you get around to it" and then later say "okay, I really need this
value right now." Adopting this approach would probably require the fewest
changes to your code.
-Eric
On Fri, Feb 11, 2011 at 12:12 AM, Aaron W. Hsu <> wrote:
> On Mon, 07 Feb 2011 20:55:27 -0500, wren ng thornton
> <> wrote:
>
> > What exactly are you looking for? And are you really looking for
> > parallelism, or concurrency, or both? SIMD, or SMP? normal computers,
> > or supercomputers?
>
> I have heard much and read little about parallelism and concurrency (I am
> interested in both concepts). Unfortunately, what I have heard is too
> general to be practical, and what I have read is too theoretical and
> abstract to be practical. The only real anecdotes I have of people doing
> serious parallelism seems to be centered around the C++/MPI crowds. I have
> difficulty fathoming that there are not other "real" users of the myriad
> of concepts which appear to float around academic paper space.
>
> In other words, say I am just a simple programmer who cares about
> systematically adding parallelism and/or concurrency to his toolbox, what
> does such a tool look like? I am trying to understand what these concepts
> look like in real world programs and what sort of interfaces are both
> efficient and pleasant to use. What sort of standard practices surround
> the use of said interfaces and what collective wisdom exists on its use;
> how does one use concurrency/parallelism in a general, responsible manner?
> I cannot imagine using either MPI or the Pi calculus directly in my
> programs, and in systems like Fortress, I struggle to extract the core
> elements of parallelism design from the rest of the languages design
> choices.
>
> I hope that makes sense.
>
> Aaron W. Hsu
>
> --
> Programming is just another word for the lost art of thinking.
> _______________________________________________
___________________________________________________
Posted on the PL-wonks mailing list. Go to http://www.cs.indiana.edu/mailman/listinfo/pl-wonks to subscribe.
