The Flow

2019

The Flow (NFN) is a -flow based model, great at .Look at our paper on noise for for some of using the NFN on real-world and .

Here I’ll the of the NFN and go some of the math. can be found in our Open- as well as in my repo.

Flows

Let’s start with the model, the final part of the NFN.We are using Flows as a way to the we are .This can model of like is means that the of (Y) is not over (X). In this for , the of Y is for (Xleq0).

and fat is one of a fat- . to a , it has that are far more (So far so that it’s mean even be )..

A Flow turns a base (p(x)) (e.g. a or ) into a more (hat{p}(y)).This is done by an , (f(x)) to (x) drawn from the base , in (y) from the .

Let’s start with a : Take a (X) to a with zero mean and .What is the of, say (Y = f(X) = frac{1}{3} cdot X)?Since we every (x sim p(X)) by 3 to get a (y sim hat{p}(Y)), we can the of Y to be more , with most being to the mean than in the base .

Now how to we the of Y? A (y sim hat{p}(Y)) is by (x sim p(X)) and then it to get (f(x) = y).For a given y, we can get the x that it by using the (f^{-1}(y) = 3y = x). We can then the of this x: (p(f^{-1}(y)) = p(x) = p(3y)).Yet that is not equal to the of our y.Since the has been , in the of y will be more than they had been under the (more out) .This can be seen in the plot, the (p(3y)) is and doesn’t to 1. we need to .This can be done by with the of the of our , (|frac{df^{-1}}{dy}| = |frac{df}{dx}|^{-1}).

This makes sense:

For , the of the the of the of the : (middet frac{ f^{-1}}{ {y}}mid).While it’s a to get an for why that is the case, in . They are to than the case since their is just the . The of a is but the of from the by the . For the 2-D case, check out this . Image the ‘M’ as a . Then the and see how the of the is to the of the .This leads us to the main of Flows which tells us how to a by (f({x})):

[hat{p}({y}) = p(f^{-1}({y}))Big|detfrac{ f^{-1}}{ {y}}Big| = p({x})Big|detfrac{ f}{ {x}}Big|^{-1}]

This can be using the -of- . (f({x})) has to be and . Many of such have been .For the NFN we three of them that we have found to be both and fast to : , and flows.We those three some of the flows like MAF or Real NVP they work well for low- .

Flows

The I used in the of this post is an of the Flow:

[{a},{b} in {R}^d quad f({x}) = exp({a}) odot {x} + {b}]

They are the Flow and can be used to shift and scale .They are not able to split a into two modes . To model , we will them with other Flows.Here we use an flow to a :

The 2-D is and , yet stays . ({a}) the of the and ({b}) is the shift .

Flows

Flows and a :

[{w}, {u} in {R}^d, b in {R} quad f({x}) = {x} + {u} tanh({w}^text{T}{x} + b)]

The base is split into two halfs in this .(b) and ({w}) the ({u}) the and of the .Side note: While (3) is using the right , no form is know. in the can only be .

Flows

flows and a point:

[gamma in {R}^d, alpha, beta in {R} quad f({x}) = {x} + frac{alphabeta({x} – gamma)}{alpha + ||{x}-gamma||}]

(gamma) is the point, (alpha, beta) how to / .

Both and flows are only if their are .How to each flow and how to the log of the is in the for my code and in and (2015).

one Flow to a in . we can stack flows to get even more flows an :

The Flow

Now let’s the Flows with a that us to model .This is to a .For the MDN the model is a of , for the NFN it’s a Flow.The NFN is used to the (p(y|x)).This is a of / . In a task for , one y given x as ({E}(y|x)). We this by a full of just a point .Say (x) is a job title and we’re to (y), the with it.For for p(wage | job=’’), this is as you can see here. Some are much off than .The NFN now works as :

A net takes x as it’s input and the of a stack of Flows, e.g. two flows and one flow (The exact setup of the flow is a of the NFN). The Flows now a of a base into a more , the (p(yvert x)). The model is then to the log of the data:[L(theta) = -sum_{n=1}^N log p({y_n}mid NN_{theta}({x_n}))]

As a small , the Flows are in the NFN and point from the to the base .This is done to speed up the , as the flows we use here we for in by and (2015). In their form they are fast at and slow at . For CDE we need the .

The NFN on a of the two moons where (X) the of (Y) the :

flows serve as a , novel way of . In the tests in our paper we the NFN the and the , among .The NFN shows on that are non-, e.g. that and heavy tails.

You can try the NFN and other on your own data using our ( 1.X) or my repo ( 2.0).

Info