Profile cover photo
Profile photo
Sergey Ten
599 followers -
Doing stuff with Deep Learning, augmented reality & 3D reconstruction, intrested in ML, nonlinear optimizaton, compressed sensing, robust statistics, AI, math
Doing stuff with Deep Learning, augmented reality & 3D reconstruction, intrested in ML, nonlinear optimizaton, compressed sensing, robust statistics, AI, math

599 followers
About
Sergey's posts

Post has shared content
Every Baby knows the scientific method
Photo

Post has attachment
Voynich Manuscript word2vec embedding shows two distinct languadges

Post has shared content
Robotic Cthulhu Overlord Santa!
Boston Dynamics is ready for Christmas yay
Animated Photo

Post has shared content

Post has shared content

Post has shared content
The complexity of integers

The complexity of an integer n is defined to be the smallest number of 1s required to build the integer using parentheses, together with the operations of addition and multiplication. 

For example, the complexity of the integer 10 is 7, because we can write 10=1+(1+1+1)x(1+1+1), or as (1+1+1+1+1)x(1+1), but there is no way to do this using only six occurrences of 1. You might think that the complexity of the number 11 would be 2, but it is not, because pasting together two 1s to make 11 is not an allowable operation. It turns out that the complexity of 11 is 8.

The complexity, f(n), of an integer n was first defined by K. Mahler and J. Popken in 1953, and it has since been rediscovered by various other people. A natural problem that some mathematicians have been considered is that of finding upper and lower bounds of f(n) in terms of n. 

John Selfridge found a lower bound for f(n) by proving that f(n) is always greater than or equal to 3 log_3(n), where log_3(n) denotes the base 3 logarithm of n. This lower bound is sharp: it cannot be improved because the bound is achieved when n is a power of 3. For example, if n=81, we have n=3^4, which (by the definition of logarithm) means that log_3(n)=4. Selfridge's lower bound for f(n) is then 3x4=12. We can write 81=(1+1+1)x(1+1+1)x(1+1+1)x(1+1+1), which uses twelve occurrences of 1, and Selfridge's result shows that there is no way to achieve this using eleven or fewer 1s. Note that we are only allowed to use addition and multiplication in our expressions; using exponentiation is not allowed.

The problem of finding an upper bound is more complicated. Richard K. Guy found an upper bound for f(n), showing that it is bounded above by 3 log_2(n), which works out at about 4.755 log_3(n). (The 4.755 is an approximation to 3log(3)/log(2).) Guy found this bound using Horner's method, which is explained in the appendix below. The worst case of Guy's bound is achieved in the case that n has no zeros when it is expressed in base 2. 

More generally, it turns out to be difficult to improve the upper bound for f(n) because numbers whose binary digits contain a very uneven balance of 0s and 1s tend to cause problems. An example of this is the number 1439, which is 10110011111 in binary. It turns out that f(1439)=26, and an optimal expression for 1439 is 1+2(1+2(1+1+3((2)(2)(2)(2)+1)((3)(2)+1))), where 2 and 3 are shorthands for (1+1) and (1+1+1), respectively. This works out as about 3.928 log_3(1439).

However, it is possible to replace this value of 3.928 by a lower number for “almost all” integers n; in particular, recent work of J. Arias de Reyna and J. van de Lune shows that it can be replaced by 3.655 for a set of natural numbers of density 1. The recent paper Applications of Markov Chain Analysis to Integer Complexity  (http://arxiv.org/abs/1511.07842) by Christopher E. Shriver improves this upper bound to 3.529 for suitably generic numbers. The paper mentions that extensive numerical computations by other authors suggest that it ought to be possible to improve the generic bound to 3.37.

Relevant links

The On-Line Encyclopedia of Integer Sequences lists the complexities f(n) of the first few natural numbers (https://oeis.org/A005245) and mentions that Guy conjectured that f(p) = f(p–1) + 1 whenever p is prime. Guy's conjecture turned out to be false, but the smallest counterexample, which was found by Martin Fuller in 2008, is surprisingly big: the smallest such prime is 353942783.

The OEIS is an extremely useful resource for researchers in discrete mathematics. They are currently running their annual appeal for donations, and you can donate on this page: http://oeisf.org/

The “extensive numerical computations” mentioned above are discussed in the paper by Iraids et al, which you can find at http://arxiv.org/abs/1203.6462

Appendix: proof of Guy's upper bound

Horner's method works by expressing a (nonzero) number n in base 2 as a sequence of binary digits a_0 a_1 ... a_k, where we may assume that the last digit a_k is equal to 1. It is not hard to show from this that log_2(n) is greater than or equal to k and less than k+1. It is also immediate that n can be expressed as the polynomial 
a_0 + x a_1 + x^2 a_2 + ... + x^k a_k
when x is replaced by 2. Rearranging, we find that
n = a_0 + 2(a_1 + 2(a_2 + ... + 2(a_{k-1} + 2)...)),
because we assumed that a_k was equal to 1. We then replace each occurrence of 2 with “1+1”, and replace each occurrence of a_i with either 0 or 1. Finally, we remove all the occurrences of “0+”. The number of 1s in the result is at most 3k, as required; the bound is achieved whenever n is one less than a power of 2.

For example, n=42 is 101010 in binary, which can be written as 0+2(1+2(0+2(1+2(0+2)))). This expands to (1+1)(1+(1+1)(1+1)(1+(1+1)(1+1))), which uses 12 ones. Since 42 lies between 2^5=32 and 2^6=64, it follows that log_2(42) is more than 5, and 3 log_2(n) is more than 12, as required.

#mathematics #sciencesunday #spnetwork arXiv:1511.07842
Photo

Post has shared content
This very depressing story has, as George Monbiot says, received surprisingly little publicity -- I hadn't heard about it until seeing his article. As he puts it,

It is surely, on any objective assessment, more important than anything else taking place today. And it shouldn’t require a columnist, writing in the middle of a newspaper, to say so. It should be on everyone’s front page. It is hard to convey the scale of this inferno, but here’s a comparison that might help: it is currently producing more carbon dioxide than the US economy. And in three weeks the fires have released more CO2 than the annual emissions of Germany.

Post has shared content
Uh-oh, shoggoths are our future...
Madeline Lancaster realized that she had accidentally grown a brain. Since the late 2000s, biologists have grown a wide variety of rudimentary organs to understand development and for medical uses.

Post has shared content
Highway networks - way to train very deep networks without freezing layers (for very deep nets, deeper than vgg)

Post has shared content
Here is a likely explanation of non-Gabor-like filters in the first convolution layer:
http://www.andrew.cmu.edu/user/hgifford/projects/k_means.pdf
Filters in the first layer are related (or just the same as) centers of clusters of whitened image patches of all the images in dataset. Clusters with big number of samples produce Gabor-like(or sigmoid-like) centroids. Cluster with small number of samples produce  non-Gabor-like complex centroids,  like filters in the layer I've posted. So it's likely non-Gabor-like filters represent distinctive features which are underrepresented in the dataset.
The appearance of them in this #convnet could be explained by higher dimentsionality  of the input, which make clusters generally smaller.
Usually #convnet  similar to AlexNet  produce filters in the first layer which are very similar to Gabor wavelets, like here

http://cs231n.github.io/assets/cnn/weights.jpeg

I thought it was near-universal rule. However here is 1st layer filters produced with AlexNet-like net, and  some of them nothing like Gabor. 
They produced on concatenation of medical images with their corner-detection filters. Some of them "N"-like, some "r"-like - completely different topology...
#deeplearning  
Photo
Wait while more posts are being loaded