#convex combination

[peeterjoot]

ECE1505H Convex Optimization. Lecture 3: Matrix functions, SVD, and types of Sets. Taught by Prof. Stark Draper

ECE1505H Convex Optimization. Lecture 3: Matrix functions, SVD, and types of Sets. Taught by Prof. Stark Draper

[Click here for a PDF of this post with nicer formatting]

Disclaimer

Peeter’s lecture notes from class. These may be incoherent and rough.

These are notes for the UofT course ECE1505H, Convex Optimization, taught by Prof. Stark Draper.

Matrix inner product

Given real matrices \( X, Y \in \mathbb{R}^{m\times n} \), one possible matrix inner product definition is

\begin{equation}\label{eqn:convexOp…

View On WordPress

[isomorphismes]

When doves cry inside a Convex Hull

You have a set with two things in it. That's your basis. A linear combination allows you to add the two things together, and to λ scale them. That's called spanning the basis.

The difference to a convex combination is:

a convex combination lets you connect two points and "be" anywhere in between

a linear combination actually opens up the whole plane that the two points lie in (a plane is a lot in 3-D but not much in higher D)

So I was listening to "When Doves Cry" and having some thoughts I could not have had without linear algebra:

maybe I'm just like my father

he's worried he's merely a convex combination of his parents--just 40% mom + 60% dad, or whatever

are people doomed to that?

maybe they could "slingshot" their way out of the trap by being "more mom than Mom" -- a more general linear combination like 220% mom + −30% dad

maybe when you meet other kids and teachers at school, you can take little pieces of them and make a new identity for yourself that way

I think I did that, to an extent; copied 10% of my step-dad's boorishness, 30% of my mom's whimsy, −250% of everything about my step-mom; copied some teachers and friends too

does that have something to do with creativity?

like maybe your band can't create a Truly New Sound, but it can choose a never-before-heard point inside the convex hull of bands that already existed.

or maybe it can push outside the convex hull of music but sounds are mostly noise, ambient, or quiet out there.

did Beefheart, Godspeed, and Cage reach out of the convex hull of existing music? From just a few dimensions outside to everything outside.

OK so a convex combination is probably not what Prince meant. But it was a related idea at least.

This flight of fancy was brought to me by: Linear Algebra Class. Mathematics is a totally different language than English. It’s more different from English than is Mandarin, Pormpuraaw, Tagalog, Aymara, Farsi, or Pirahã. That means you can think different thoughts once you learn mathematics. You can fathom what was unfathomable.Conceive what was inconceivable. See what was invisible. It also means that learning to “speak”this way sounds very strange.

[isomorphismes]

Convex Combinations

Jack Sprat could eat no fat; his wife could eat no lean.  And so, between the two of them, they licked the platter clean.

With my girlfriend and I the meals are not divided (100%,0) or (0,100%).  But the same concept applies:  I'll have 25% of her beer and she'll have 25% of mine.  The nursery rhyme stands in for the general idea of a general convex combination -- any such combination as (53%, 47%), (1%, 99%), or (25%, 75%).

That's what a convex combination is.

It's written with a λ and looks so much more mystifying that way:

But just say that A and B are two things, like in the case above, two 4-D vectors each containing (amount of Guinness I have;   amount of Guinness she has;   amount of Old Rasputin I have;   amount of Old Rasputin she has).  The quantity

mustn't total up to more beers than we bought ... which is common sense, really.

Wax Philosophical

So if the definition makes sense, let me just throw out a few mind-expanding ideas you can conceive with it:

Mixing colours is a convex combination. (R, G, B) is a linear 3-space. So is (H, S, V) -- and too, there is a reversible transformation from one to the other. (C,M,Y,K) is a 4-space so the transformation can't be so simple.

Can you then say that one colour is "between" two others?

Can you imagine a colour that's a convex combination of three colours? Would that make sense?

On from colours to ideas. Have you ever noticed that if people are taught two competing theories in a class, then they try to balance between them? I noticed this in political theory, anthropology, and philosophy classes.

The Economist's Which MBA ranking allows you to adjust the importance of various factors. Typical college ranking systems do the following: (1) observe and score schools on several facts, (2) combine these (independent or not) dimensions into a total order (3) using the weighted-average method. The weightings are arbitrary, which mean the ranking would be different for someone with different priorities. If I assume your preference weighting is linear then .

I have a pet theory that it's very natural for people to want to compromise among the ideas that they're given -- i.e., occupy some convex combination rather than a "corner".

My pet theory goes further to say that revolutionary ideas don't necessarily have to be "orthogonal" -- don't have to be completely radical and unintelligible according to current ideas -- to permit novel thought. If the idea has even just a little bit of a unique notion (points just a wee bit into a new dimension), then that idea can be combined, linearly, with old ideas, and the entire dimension of new ideas is opened up.

Lastly, science. You can have a convex combination of quantum states. That's where the concept of superposition comes from.

From a geometrical perspective, convex combinations happen on the surface of a high-dimensional sphere, restricted to the positive "octant" i.e. all angles between [0°,90°].