Composition Lib Blabbing, #kde-multimedia, August 31, 2005

wheels on the other hand is en-route to going down the road that eventually seems to side-track all KDE multimedia developers over time -- I started playing with ideas to write a synthesis lib. :-)

<kampfschlaefer> wheels: if it isn't supporting ladspa/dssi, I won't get attracked by it ;-)

<kampfschlaefer> and you need an IDE for your synthesis-lib just like artsbuilder :-P wheels suposes he should actually try artsbuilder again...it's been a *long* time.

<wheels> But I think my goals are fairly different.

<kampfschlaefer> it hasn't changed since then...

<kampfschlaefer> wheels: take a look at ams, galan and om

<wheels> kampfschlaefer : right, but I've changed since then. :-)

<wheels> But I'm actually not thinking about a UI at this point. Actually this is more in frustration with UIs...

<kampfschlaefer> that leaves om to look at. afaik it has guis which connect via OSC but can work without...

<wheels> What's the expansion for ams? :-)

<wheels> The other two I can already tell very quickly aren't what I'm looking for.

<kampfschlaefer> ams = Alsa Modular Synth, unfortunately it is dead/on-ice since Matthias Nagorny is busy with other things at work...

<kampfschlaefer> but if galan is not for you (except from the gtk-ui), ams isn't either...

<wheels> I suppose I probably didn't describe well what I'm interested in --

<wheels> Really I want a set of C++ (or other sane OO language) for doing music composition. There's a Java lib that goes in that direction, but I wasn't terribly impressed with its API.

<wheels> I have two problems with most of the graphical systems:

<wheels> (a) The semantics are very weak.

<wheels> (b) The flow of time is very ambiguous.

<kampfschlaefer> wheels: search tha archives of LA[ADU] there where some announcements that could interest you. something about synthesis-libs, I think there where even in c++

<wheels> What I'll more likely do would be write a scripting interface for jMax / Pd, but I'm just playing around with some ideas at the moment.

<wheels> kampfschlaefer : the problem with most of those things is that they're written by audio people, not musicians, so they don't really lend themselves to working with them compositionally.

<wheels> I know that sounds strange, but essentially what I want to do is to write a "score" in a programming language.

<kampfschlaefer> wheels: do you want synthesis or arrangement?

<wheels> What I really would like is something where I can do things like: "start this clip at beat 4, loop it three times, each time applying this filter".

<wheels> kampfschlaefer : both, actually.

<wheels> kampfschlaefer : but that's the problem -- to a composer they're not separate problems. To an audio person they're completely different.

<kampfschlaefer> because the audio-person also thinks about the musicians, which don't write the score but read them and play their instruments with their individual sounds...

<wheels> kampfschlaefer : no, no -- you're abusing the work "musician" in the same way that most audio people do. :-)

<wheels> "musician" isn't someone that plays an instrument...

<kampfschlaefer> wheels: hey, I am everything except the real composer myself...

<wheels> i.e. a "composer" is a "musician". :-)

<kampfschlaefer> okay, /me thinks that depends on the music you make

<wheels> a violenist is also a musician, etc.

<kampfschlaefer> in my view the one playing it isn't necessarily the one who write it...

<wheels> Well, someone in the symphony wouldn't say that the composer isn't "one of the musicians" -- that's the way the term gets abused in audio engineering circles, but musicians don't use the term that way...

<wheels> kampfschlaefer : that's right -- but the one playing it and the one who wrote it are both "musicians".

<kampfschlaefer> so musicians includes composers?

<wheels> sure

<kampfschlaefer> okay, term defined

<wheels> So when I say "the interfaces aren't musician friendly" what I really mean is that they're not set up for creating music direclty -- they're organized at the wrong levels of abstraction for working with musical ideas.

<kampfschlaefer> but if you don't want to distinguish between the writing and the playing of the "score" you have to distinguish between the arangement and the synthesis?

<kampfschlaefer> ie. if someone else should play your pieces...

<kampfschlaefer> :s/don't/do/

<wheels> Well, a score doesn't have a 1-to-1 mapping to sound in most cases. It's a tool for expression.

<wheels> i.e. a score represents a musical idea

<kampfschlaefer> but you are searching for a tool/lib that ties both (score/idea and interpretion) together

<wheels> Basically what I want to do is to be able to work with musical ideas using a programming language to express those. From the technical side that may mean setting up certain parts which react to input during performance (either from external audio or MIDI input), but you also have things like randomness that are used in electronic composition...

<wheels> Really conceptually something like MAX is the right direction -- I just think that it's a pretty bad implementation of the concept.

<wheels> Going back to my original statement -- I think something with stronger semantics and a clearer flow of time would be helpful. I'm not sure what that would look like in a GUI.

<kampfschlaefer> like beast? synth and "sequencing" integrated into one?

kampfschlaefer is still clueless howto represent time if not with a timeline

<wheels> Nah, they're still basically two classical interfaces just put into the same application.

<wheels> That's why I said that I'm not sure what a UI would look like. But I don't really care.

<kampfschlaefer> it would require a 3d-interface if you want to add the time to a 2D-interface of a synth...

<wheels> Since I'm not primarily concerned with coming up with a UI...

<wheels> Well, I think it would require some sort of new UI model, but I don't think it'd need to be 3d.

<wheels> I'll give a practical problem with a MAX patcher:

<wheels> You set up your elements by wiring them together; but some of the elements naturally delay the signal. So then you have two branches that reconnect later in the patcher -- but it's not clear what the state of the two signals will be when it reaches there. You have to do a lot of extra work to try to get two lines synchronized based on the original input.

<wheels> i.e. when writing the synth last week I hit the problem that I was running the MIDI input into a sub-patcher that tried to keep track of which voices were in use in an array that mapped MIDI pitches to oscilator banks (monophonic synths), but running it through that sub-patcher added enough delay that when I tried to use the output in conjunction with with velocity information that I also got from the MIDI input they were out of sync.

<wheels> So I'd get the velocity for one note with the note from another.

<wheels> err, with the pitch from another.

<kampfschlaefer> from the programers view I would say: why did you split information belonging together into parts? wouldn't it be better to handle the note-events as whole with pitch and velocity tied together?

<wheels> Yep.

<wheels> That's what I wanted to do, but couldn't find a way to do that. But I'm not sure that packing / unpacking elements are enough to solve the problem in general.

<wheels> (though it would solve this specific problem)

<wheels> http://ktown.kde.org/~wheeler/images/poly-jmax.png <-- there you're not sure that the two outputs to the "poly 4 4" element will happen at the same time. In fact, they don't.

<wheels> err, three outputs.

<kampfschlaefer> is that compensated by the "sync 3"?

<wheels> I hoped it would be, but apparently not. :-)

<wheels> I'm sure that I could (and will) get it working properly, but the amount of trial and error in working with jMax has kind of led me tot he ideas I'm talking about. :-)

<kampfschlaefer> :-)

<wheels> Later tonight I'll give it a whirl in Max/MSP and Reaktor to see if things are any better there. :-)

<wheels> http://ktown.kde.org/~wheeler/images/full-synth-jmax.png <-- the rest of the synth, at present. :-)

<wheels> But also that the ins and outs aren't labeled or typed kind of bothers me. I'm not sure that if you're working in a graphical environment with complex components though if it's practical to do so.

<kampfschlaefer> yeah, that bothers me too whith pd...

<wheels> But I'd guess that just the stuff to do a basic synth like that would be at the most 5% of what would be involved in working out even a fairly short composition in Max, and it's already a huge pain to work with.

wheels should do a talk at the next LAD conference: "Thinking Like a Musician for Engineers" :-)