Profile cover photo
Profile photo
Blaine Bateman
a fusion of engineering and art...
a fusion of engineering and art...

Communities and Collections
View all

Recently spent a weekend getting public economic data into R via APIs and web scraping. I use such data in forecasting models for B2B clients. The US Federal Reserve and Census Bureau and Economic Information Agency provide a rich set of historical and forecast data products. I suspect that in the next decade many of these sources will no longer be maintained and refreshed due to budget cuts and downsizing.

Have been working on combining linear regression with neural networks as a post-processing step, to try to leverage the speed of linear regression with the ability of the neural network to model nonlinear and interaction effects. I hope to post some results soon.

Post has attachment
Add a comment...

Post has attachment

As many of you know I have expanded my practice to include analytics, including using machine learning and regression methods on time series forecasting. Time Series models are essential to many business predictive use cases, although if you follow only the popular press or trade media you may conclude that deep learning and classification models are all you need. While many business decisions can be informed by deep learning, nothing could be quite as fundamental as forecasting sales for a business.

If you research time series modeling you will likely be led towards a class of approaches using "autoregressive" methods. You may find things like ARIMA (Autoregressive Integrated Moving Average) and variants. This was unsatisfying to me as such models are effective pattern-finders but do not incorporate any business or domain intelligence into the model. Thus I focus on time series regression models where at least some of the model depends on business variables, such as past sales, pipeline, inventory, lead time etc., as well as exogenous variables such as economic factors, exchange rates, weather, etc.

I will write a series of notes regarding some of the knowledge gained in the past several years, as well as recent work. My first note will review tools.
Add a comment...

We have decided to make our Silicon Photonic work available at no cost. Send me a query and I'll send it to you.

Post has attachment
We have released our newest work in the emerging area of Silicon Photonics. Those interested to obtain the work can go here:

We are offering the work at $499 which is unheard of in syndicated research; we are doing this to enable more people to obtain the work, including universities and small businesses. We are pleased to provide the Executive Summary and Table of Contents here.

Silicon Photonics is the science and engineering used to leverage silicon semiconductor foundries to access to the vast knowledge, experience, and installed capital base in semiconductor manufacturing to produce photonic integrated circuits (PICs) with high yield, and low cost. As used in reference to commercial devices, Silicon Photonics is a finished product that includes use of silicon PICs and whatever else is needed to complete the desired product. The "modern era" of Silicon Photonics, meaning significant level of component integration in a single PIC, dates from around 2000; in the following 17 years, thousands of universities and dozens of companies have contributed to the technology, investing 10s of billions of dollars.

This work is intended to provide a detailed understanding of the Business Situation regarding Silicon Photonics. This is not a typical market research report, in that we do not develop market forecasts nor slice them across various market dimensions. While many companies, universities, and other laboratories are mentioned, in general we won't develop detailed vignettes about them. Instead, we focus on the technology status, the many options open, and what that means to participants in the value chain.

In some cases, Silicon Photonics is taken to imply the silicon PIC is directly integrated with electronics to make a complete transceiver or other product. However, even research examples of that level of integration are rare, and full integration in a single wafer flow is very unlikely for some time, due to differences in the process node size used by Silicon Photonics (~65 nm node) today vs. state of the art electronics (14 nm or 10 nm node), among other reasons.

Complete products based on Silicon Photonics thus require downstream integration, and there are some novel processes demonstrated such as integration on an optical interposer, or another substrate. In any case, chip to fiber coupling is a high risk area, having been demonstrated in only limited high-volume cases. The optical interface also points out the need for active optical and electrical test, which creates additional work flows compared to electronics. Additional process flows, downstream integration, test, and final packaging all add cost and risks. These seemingly simpler problems are viewed by industry experts as the main barriers to large-scale adoption of Silicon Photonics.

Although it is becoming rare to read a trade article having to do with data centers or the cloud without some reference to Silicon Photonics, we will motivate in the narrative that Silicon Photonics is far from mature technology, and that rapid evolution and improvements will occur over the next 10 years. The nascent state of Silicon Photonics technology creates risks to not only designers and foundries, but also early adopters. This manifested as the risk of paying too much for too little performance, impacting capital investment, capacity, or operating costs negatively compared to next-generation products. In the hyper-competitive cloud services market, decisions to go with new technology must be made with diligence.

We summarize the risks in a Risk Meter:

We arrive at the above by looking across 11 key elements needed to deploy a Silicon Photonics solution and across the perspective of various stakeholders from Silicon Providers to Cloud Consumers. We conclude there is a level of risk that raises some concerns, especially for silicon providers (including fabless), packaging houses (or equivalent in a vertically integrated supplier), and early adopters.

The underlying driver for all the investment and work in Silicon Photonics is the relentless increase in network and data center capacity demand, in turn driven by the internet. The challenge operators and providers are facing is that higher and higher data rates, from the chipsets to the intra-data center switches, are exceeding the limits of copper-based communication. Initially, the demand was met by migration to gigabit Ethernet, then 10 GbE, and now 25 GbE.

At 25 Gb/s on a single channel (or copper trace, or single fiber) even the data movement in the server boards and backplanes is challenging. At the same time, data centers have become much larger, and the physical distance of the longest interconnect may be over 1 km. All of these factors make a transition to optical communication imperative and inevitable. It has already begun in many cases, and the challenge is now to create lower cost, smaller, lower power, and higher performance ports to connect from the severs on up in the data center fabric. 100 Gb/s in a 4x25 (4 fibers, 25 Gb/s per fiber) configuration will rapidly become the standard at the server backplane. At present, Silicon Photonics is viewed as the most likely solution to these challenges in cloud-scale data centers.

There are several key points that should be kept in mind while reviewing this work or researching the Silicon Photonics market.

Relentless increases in data traffic drive huge increases in data movement within cloud data centers, pushing interconnect to higher speeds, already moving from 10 Gb/s per channel to 25 Gb/s per channel. Given the physical scale of cloud-scale data centers, at these speeds, photonics is the only feasible solution at present.
Before long, even data within the servers will need to be moved on photonic channels, and eventually from chip to chip. Silicon photonics is viewed as a path to integrate the needed photonic functionality with electronics without incurring large cost penalties and while lowering energy consumption.
Silicon Photonics can enable a new design paradigm for data center and server design, including a flatter network architecture. This implies some products are more likely to appear in new data centers sooner than upgrades, although upgrades will certainly take place.
Once a generation of photonically enabled servers is commercialized, costs will fall rapidly. In addition, prices for "traditional" data center hardware may fall in response. Enterprises and data center operators will need to make decisions even while the technology is still evolving.
Whether you are an IT consumer of hardware or services, or in some other part of the value chain, this report will help you frame your thinking and reduce the risk to your strategy of being unaware of possible huge shifts.
Important ideas developed in this work include:

Silicon Photonics in some form can disrupt current on-offer data center interconnects, including some existing optical solutions.
The current market leaders are in good positions to ride the cost curve down somewhat, but companies like Intel and IBM may be better positioned because they have vertically integrated fabs and broader product offerings.
Over the last 10 years, a range of EU programs have been funded specifically to advance commercialization of photonics technology, including Silicon Photonics. One result is that STMicroelectronics is one of the few Silicon Photonics-capable foundries available.
A significant share of the market likely will be served by fabless companies leveraging commercial foundries. The barriers to this are limited foundry capabilities and design kits available at present.
Intel, IBM, NTT, Cisco, Fujitsu, Oracle and others are likely looking at long-term roadmaps with more integrated photonics of their own designs or sourced, including possibly at the chip to chip level.
Existing interconnect specialists are in the situation of either controlling their own destiny by adding Silicon Photonics offering, or watching prices get crushed anyway.
Fragmentation of the market poses a challenge to get enough share of the TAM to drive economies of scale.
There is nearly frenetic level of activity in the R&D community and in many start-ups. This is exemplified by the following figure, which charts the publication of papers related to light sources (lasers) for Silicon Photonics.

In conclusion, we think that Silicon Photonics has earned a place in the next generation data center optical communication solutions. The technology is ready to begin a life cycle similar to the Moore's Law trajectory followed by electronics integration. We expect rapid introduction of new solutions in a very competitive market for the next 10 years, or more.

In addition to the narrative analysis, we provide a timeline history of developments in Silicon Photonics, containing 446 entries from 2000 to 2017. All material used in this work is cited in the included bibliography, containing 792 entries with author, title, and web links.

All players in the Silicon Photonics value chain should be making informed decisions today about how and when to participate, from fabless design houses to communication specialists to data center architects. We sincerely hope you find this work enabling in your decision making process.

Blaine Bateman, President, EAF LLC

Table of Contents

Executive Summary
Six years of "almost there"
Focus on Data Center applications
Metrics show frenetic activity
Business Implications
Omnipresent Photonic Interconnects
Applications and Architecture
The New Moore's Law?
Market Motivation
Let There be Hype
Value Chain Implications
Summary of Implications
What This Technology Means to You
The Elephant in the Room: Why do thks?
Key Points 1-6
Photonic Integrated Circuits
Light source integration
PIC Material Platform
Advanced modulation formats
(Too?) Many Material Choices
Silicon Photonics Technology Development—Status and Risk
Data center fabric
Maturity Metrics
Not just Silicon
Fab node paradox
Manufacturing (fab process
European leadership
Rest of the World
The Promise of Fully Integrated Solutions
R&D perspective
Light Source
A wide range of possible solutions
Light from Silicon
Wavelength multiplexing
Direct modulation
Rings and racetracks:
Gratings, polarization, and other approaches
Polarization effects and control
Optical power loss challenges
Beyond Si-Ge APDs
III-V integration additions to CMOS flow
Group IV lasers
Monolithic solutions (with additions to CMOS flow)
Chip to fiber coupling
History of Silicon Photonics: Timeline (446 entries)
Bibliography (792 references with links)

Post has attachment

Post has attachment
This is not new but on point and funny.

Scare tactics

It is interesting to consider marketing and sales hype around various topics through the lens of Michael Porter's 5 forces (now 6) model. In fact, it is the 6th force, complements, that applies here.

I follow a number of topic areas including Internet of Things; mainly in the Industrial (more properly: non-Consumer) space (IIoT). As the hype of IIoT has grown over the last several years, the deployment of IIoT has become a very strong complement to those selling security. Security here is mainly in the Cyber sense. The level of alarmist articles regarding un-secure IIoT systems has been increasing in volume along with the IIoT hype itself.

This is one of the best examples I can think of for complements--without the IIoT, there would be no need for IIoT security. The demand for IIoT security is correlated to IIoT deployment.

I think that complements are most effectively used when considering a new market area in which to enter. If clear complements exist, then it is possible to look at the market dynamics of the complements as part of the analysis. A market where the complements are declining, such as desktop computers as complements for sales of mice and other similar devices, could be a poor choice, even if there is pent up demand for a new disruptive product. On the other hand, a well designed strategy to take advantage of the demand bubble but not over-invest in the long term could be appropriate for products in the desktop space.
Wait while more posts are being loaded