Friday, September 18, 2009

When Google Gets Musical

Here in China I’m amazed by the Google Music site... I can search for all kinds of bands and musicians and get their albums, with songs fully playable in an easy interface (songs playable in China only, that is!). The scope of featured artists doesn’t feel unlimited but is still quite broad. If you’re interested how Google Music China came about, below article by Michael Zhang appearing earlier this year in Chinese was kindly provided by Michael. The translation was provided by Michael’s colleague and is presented here often “as is,” so please forgive potential slight ambiguities or translation barriers here and there.

Michael Liang Zhang is the assistant managing editor of Global Entrepreneur magazine. He writes a blog focusing on Apple and is following Google China’s story for several years now, interviewing many of Google China’s employees.

“What should the ideal music product look like?”

“Whatever you think of will immediately play,” Hong Feng, product manager of Google China, answered. Perhaps not fully satisfied with the answer, he adds, “It will play the right music without you having to give it any thought.” You get the music you want at the right time, for the right environment, and in the right mood.

Well, it sounds unrealistic that one’s thoughts can actually control music. Yet like the famous science fiction writer Arthur C. Clarke said, “Any sufficiently advanced technology is indistinguishable from magic.”

Revolutionary technologies like electrical power, airplanes and search engines have all changed the world in ways that were out of expectation. To some extent, Google’s second edition of their music search product, launched on March 30 by Hong Feng and his team, fulfilled the criteria in some way. When Google China’s president Li Kaifu [who by now quit Google] and executives from hundreds of record companies posed for a photograph at the media conference, people might have ignored the fact that this product transcended reality on at least two levels.

Google and Top100.cn provide complete music archives of over 140 record companies for download, which are totally free of charge and without Digital Rights Management (DRM). The product changes the way people interact with music. If search engines reduced the cost for people to find information, and community websites flattened peoples’ relations of “the six degrees of separation”, Google Music unprecedentedly enriched the way people find music. You can find a song through the name of artist, titles of the song, albums, or even a sentence of lyric, and you can also play the hottest songs from the charts. However, the most impressive breakthroughs are these two functions – one can have music recommendations according to difference of the tempo, tone, and timber; similar songs are recommended according to the timber of specific songs. Fresh experience it offers and the technical complexity in its realization makes it the most ambitious and imaginative work of Google after it entered China.

However, there’s still much room for improvement. Though the contract with record makers is only limited to the Chinese mainland, the record industry would not like to stick to this fixed pattern all along. “If it is proved to be a successful model in China, I would be surprised it won’t be promoted worldwide,” Sandy Monteiro, vice president of the Universal Music Group told Global Entrepreneur. Though no one knows how much time it will take, we might as well just imagine, some day in the future, when this free music service born in China enters the U.S. market, how would the largest online music stores – Apple’s iTunes App Store – have to compete?

Google’s previous attempts of developing a music search service were all aborted due to disagreements inside the company. Besides the copyright issue, Google’s decision makers were not really convinced that the proposals would help it surpass its rivals in this field. Then why did such a music search product pop out in China, where either technology innovation or property protection prevails?

Forget about the highly conclusive answers. Like all other innovation process, the course is complicated and not easy to define. You can also sum up with the timing, geographical and human reasons, yet it is no other than a series of consequences of human actions: the wildest fantasies, most carefully cultivated proposals, detailed communications inside the company, hard work of hundreds of thousands of codes to convert physical sound into arithmetic, and also some good luck ... all these factors combined have accomplished an “impossible mission.”

The Google Music homepage. Outside of China you can see it, but you can’t playback the songs.

Expanding the category for European music.

An album display. You can add a song to your current playlist, which is kept open in a second browser window, by clicking the plus icon. You can also find similar songs to the one being shown (with varying results quality).

The playlist. You can drag and drop items around. All songs can be fully played. A link at the bottom reminds you of the service’s terms. Many songs have lyrics coming with them to read along. You can also order ringtones for certain songs. A banner ad is displayed at the bottom of the window.

The “song screener” lets you find songs by adjusting slide rulers in the Flash app... if you can read Chinese, that is.

Tyranny of the “or”

It never has been a secret that Google China needed a music search product.

But it is also evident that, by normal business logic, search engines and music industry are run in totally opposite directions. While Google believe that all information has value and should be found with the lowest cost, record companies believe that the limited valuable information should be kept slightly and be charged for use every time. When Google is dedicated in digging out any information hidden in corners, record industry is busy contesting each other by spotting music genius in billion.

Different orientation decides that interaction between the two sides would hurt each other’s benefits. In China, Baidu is a successful example. It takes 60 percent market shares of search engine market, of which 15 percent is contributed by music search. Netizens can get large quantity of free music for download. The only problem is that the music has no copyrights. Record companies can hardly make profit out of it, which has constantly brought disputes between each other. But Baidu insists that the music they provide for download already existed on the internet, which helped it avoid legal responsibilities.

Google China did have such kind of temptations. In late 2006, Baidu won a lawsuit against International Federation of the Phonographic Industry. Right after that, Google purchased the video website YouTube which is also bothered by copyright disputes. It is at this occasion that staff of Google China suggested: shall we develop a music search product the way Baidu did?

But this hasn’t become an option. “There is a saying in English called Tyranny of the ‘or’, which means that people can easily get themselves stuck in a kind of confrontation between two opposing options – you might as well choose to make it legal but with fee, or rather, free of charge but illegal. We don’t want to surrender to this tyranny of the ‘or’. We made it, with free and legal music,” Li Kaifu told Global Entrepreneur during an exclusive interview.

The first problem they should solve then was how to convey their sincerity.

The difficulty for internet industry and record industry to cooperate does not only root in their different positions, but also in communication difficulties. The two sides speak different industry languages, and the aggressive lawyers arguing for their legal rights made the negotiation break sooner than you expect.

In September 2006, Google China’s then chief strategy officer Guo Quji was introduced by a lucky chance to Chen Ge, the founder and CEO of Top100.cn which was invested in by basketball star Yao Ming. According to engineers’ impression, Chen’s style of eloquence would not make things any better. Yet Guo found the other side of Chen. “He has character of a ‘preacher.’ If you are running late for the appointment, he would be waiting for you at the entrance of the company till you come. If he writes you an e-mail and doesn’t get response, he would continue to write another the day after in politeness.”

As it turned out, Chen’s emotional quotient is out of expect. He never lost temper once through the whole process of negotiation, and he could always stay calm and find consensus for both, which made him sort of insulation that eliminated contradictions lying between Google and record companies. Before founding Top100.cn, Chen set up the company Pulai Music, which had made records and organized concerts for rock star Cui Jian. This made Chen familiar with the language of the music circle, which qualified him for the role of interpreter between the two sides. Chen and his Top100 has never provided music without copyrights. The two sides started with the same “no original sin” gesture into the record industry.

Under Chen’s recommendation, Google China contacted Sony, Universal, and EMI music respectively in November 2006, December 2006 and January 2007, which was right at a turbulent time for Google China. Li Kaifu, who was then reported to step down, led his team and made it clear the cooperation with Top100.

However, Li’s decision cannot stand for Google. Decision for such a product should be reported to Google’s U.S. headquarters, approvals from the two founders and the CEO are needed.

How to make this product proposal, which had been constantly denied, prevail this time? It should give credit to Google China’s wisdom of a roundabout way in communication. They head on with permits authorized within the mainland scope, and reported stage accomplishment and kept delivering the message that what they were working on was worthwhile. Trust from the headquarters was accumulated.

The first step was to make Top100 part of Google China’s interest. There was no doubt that Google had to invest in Top100 if they planned to carry on the music search business in long term. In February 2007, Li Kaifu, and James Mi, then in charge of merger & acquisition and investment of Greater China, reported this purchase plan to CEO Eric Schmidt. They didn’t boldly announce the whole music search program, but just illustrated the importance of this deal, and by this chance, introduced the meaning of music service to Chinese users. Schmidt, who had always been supportive for China business approved the plan very quickly.

It was not until November 2007 that the music search program was finally reported to the two founders. During the past several months, the team for music search in Google China has developed bold paths – cooperation between Google and Top100 had been divided into six steps. The Onebox plug-in among search results launched in August 2008, and the search page launched on March 30, 2009 are only the first two steps. [Checking the archives I can see the Google Music page went live around August 2008. -Ed.]

“The co-founder Larry Page only asked one question and signed the project with CEO Schmidt. The whole course took only a few minutes. This became a little legend for Google China.”

The biggest challenge was that Google had long defined its global strategy as search, ads, and apps. Music search should not become an area to be devoted with much energy. But Li and his team also made a convincing explanation: the content would be provided by Top100 and Google China only acts the role as platform, which doesn’t go against the search-dominating strategy; in addition, the lack of a music search product might make them lose loyal users. As a backup plan, they even proposed to buy a new domain m.cn, hoping that it would help lighten the color of Google.

The co-founder Larry Page only asked one question and signed the project with CEO Schmidt. The whole course took only a few minutes. This became a little legend for Google China.

A destined miracle?

Even with the permit, Google China’s investment in Top100 had been very cautious. Mi drafted a very detailed investment structure. Millions of dollars investment had been divided into four rounds. The second round of investment came only after Top100 signed another record company. As it turned out, Chen is a very reliable person. By September 2007, he had signed agreements with three record companies including Sony, Universal and EMI for audition of DRM music free of charge.

Details of the negotiation that had not been revealed officially reflected wisdom of Google China’s pattern in cooperating with Top100. For instance, to avoid hasty in cooperation, Top100’s agreement with record companies didn’t limit to advertisement shares. According to an insider, every contract Top100 signed with record companies was very complicated. Apart from the ad dividend, record companies can get fee for download of each song and base revenue for each year, all paid by Top100.

Apart from that, record companies had chance to get options in Top100, which put the website into a virtuous development cycle. For one thing, record companies then had more patience for Top100’s product. For another, more shareholders brought hope for Top100 in developing into a leading force in China’s music market. The expectation that the website will lead a dominating role some day was irresistible to record companies for better financial payback.

However, the most important thing is to change the idea lying deep in the music market.

Digital music brought convenience in duplication, and time of relying solely on a single album had gone. Artists have more options such as representation and concerts. Record companies are also changing their role from making music to marketing and promotion.

This was just where Google’s negotiation team hit the point. By putting forward the idea that CD works only to loyal fans while a search engine is a better choice for artists to promote themselves. A search engine also has the advantage in assembling music fans located in different parts of the world, which for certain lowers the cost in targeted marketing. [Note Google Music China uses geolocation to ban IPs from other countries. -Ed.]

Google itself could serve as a good example. In 2006, the second year Google opened China market, it made a revenue of 100 million dollars through its 20 percent shares in China. And according to Chen Ge, Chinese record industry made altogether 50 million dollars in 2008.

A standing obstacle was the issue of DRM that limits copying of the music and broadcasting might be technically strained, which would bring inconvenience to users. In October 2007, Google made a clear statement that Top100 must get authorization of the music without DRM.

Google’s music search intended to be launched on March 2008 was postponed several times. The time from October 2007 to May 2008 was called “the dark period” – no one could ensure their vision could be realized, and even Chen Ge himself could not bear this pressure.

“[T]he manager made promise that he would lift the DRM standard before leaving ... The door to the music market was finally opened, by the gatekeeper who was about to leave his position.”

It was suggested that Top100 invent a new copyright management standard affiliated to the DRM, which wouldn’t affect user experience in China. Yet the idea of borderline practice was denied by Google which had a clear record. There was still one way left – to provide a streaming media player, but Google gave up the plan, for the lack of download function might destroy user experience anyway.

The turning point came on May 10, 2008 when Chen Ge and Guo Quji visited the digital music section of EMI. At the Abbey Road Studio where albums of Beatles, Queen and U2 were recorded, the manager announced that he would lose his job in about a month. EMI had spotted Google’s chief information officer Douglas Merrill to take his place. Yet the manager made the promise that he would lift the DRM standard before leaving.

The door to the music market was finally opened, by the gatekeeper who was about to leave his position.

EMI’s attitude brought a chain reaction from other record giants. In spite of these progress, it was not until in early 2009 that Li Kaifu finally got consensus with Sony, which owns 40 percent of Chinese-language songs.

“It is a miracle that we finally made it,” Guo Quji who had left Google and started his own business told Global Entrepreneur. It took Google two years to turn the idea into reality, and lift all the obstacles. Many involvers, in Google or record companies, had left, but the deal was accomplished eventually.

Maybe the success was destined. Apart from traditional business wisdom, some wild illusions were needed to fulfill this impossible mission.

Assembly

On August 5, 2008, in an office named “Kai Xin (in Chinese: happiness) of Google China, people were not in mood of happiness.

Google’s first edition of music search was about to be put online. The music search development team had increased from one engineer to 10 people, which was not a small team in Google. They had gone through the long and lonely period of waiting: after March 2008, the product might be released every month, but was postponed each time. It was depressing that they might have to wait for another month, through the excitement of the national Olympics feast.

According to the plan, they should put the product online on 10 o’clock. If everything went well, they might go out for lunch and celebrate.

But accident came sooner than they expected. They were freaked out that Chinese users cannot visit the page while feedback from India and New York was coming in. According to the plan, Google Music was to be opened just to China, and recognition of IP address would help that. This accident might become a laughing stock, and what’s even worse, they might lose trust from record companies for this matter.

Not knowing what else to do, the team took the product offline and checked every line of code. While word spread on the internet that Google launched its music search, the project managers didn’t even know when exactly the product would finally be put online. “If things go well, it might be ready in half an hour. If not, 12 hours would be also possible,” said the product manager Hong Feng. Hong was sitting for hours in front of the video conference camera at Shanghai office, blank in his head with his face spotted on the wall.

It was already eight o’clock in the evening when they finally found out the problem – code conflicts with another product – and solved it.

It should not be overstated if we say it is a product that suffered with many setbacks. “We had many similar moments in the past year and half. We all acknowledge the difficulty and impossibleness lying in it. What we have is the bold determination to make it happen,” Tang Ting, the fronted technology leader of the music product said.

Here is another side of the story. When Li Kaifu, Guo Quji and Mi Qun were taking efforts to clear the road, these engineers were working on building a bridge for users to get better music experience. For a very long period of time, everything seemed uncertain, not only the product’s destiny but also its functions. It was just due to this kind of uncertainty that helped create a big surprise in the end.

The first person involved in this program is the product manager Hong Feng. Guo Qujin felt that Honog is a very good person to talk to when the product was still an illusion. Hong has this pure curiousity for things, and when the things come into shape, he pursues perfection of every detail.

“To avoid developing a product that annoys users is easy. But creating a product that doesn’t make you unhappy isn’t equal to creating one that makes you happy!” Hong told Global Entrepreneur. He spoke slowly and always made metaphors. He likes to make expressions of thoughts more vivid, which might have relations to his art background in primary school. He likes to make the product idea visual too.

When he imagined that all music of mankind is digitally formatted, he felt himself floating in the sea, with his hands reaching out for bubbles of songs.

Yet how to let a man under the sea get able to reach the bubble they want? This is a problem for engineering majordomo Lin Bin and his team to solve.

Lin is obsessed in telling his experience of joining Google. In September 2006, Lin, then working at Microsoft Research, met his old acquaintance Li Kaifu on a business trip, and asked Li, “why don’t you consider music search?” “Sure we would,” Li agreed. “Why don’t you join and take charge?” Lin joined a few months later.

Lin selected three fresh employees to joint this “Project M” program. Tang Mengya, a lively girl from Hang Zhou, and Zou Zhensheng, who is timid but loves singing, took charge of how the product would be presented. Zhao Qisheng, known for steadiness and patience, was devoted to back-end database and communication with Top100. In the earliest three months, they didn’t even start writing code, awaiting permit from the founders. Even Li Kaifu recalled that the engineers were in “the most pathetic” position. They still had too many obstacles to clear. To avoid frustration in the team, Lin had to communicate in a way of “reporting the good news but not the bad.” When Lin looked back, he was very grateful to them, “They knew how much uncertainty there was.”

When the program was finally approved, Tang Ting, who was then working on video search at Google China, offered to join. Tang grew up overseas and when he applied to come back and work in China, he was intended to do something with more Chinese characteristics. His interest for this program also came from people’s criticism of China’s lack of copyright protection. Tang brought in a big Northeaster Han Zhun. As it turned out, he also brought carefulness in correcting code with patience and strength.

But they still needed one person to bring technological breakthroughs for the product.

In early 2007, Li Kaifu made a speech of Google China in the New York office, hoping to attract more top engineers back to China. After his speech, a delicate girl found him. This female engineer in New York office was Hu Ning, and she was also a schoolmate of Li in Carnegie Mellon University. After a short chat, Li suggested that Hu Ning meet with Beijing’s Lin Bin.

When Lin Bin and Hu Ning finally sat together and talked, Lin found that Hu participated in a music search product when she was an intern in Google back in 2004. Hu also studied in frequency processing and message retrieval. This was a big surpise for Lin. Hu agreed to move back to China and devote scientific and technological achievements to Google’s music search.

Play by ear

When you get large quantity of music, how do you deal with it?

The simplest way is to change it into a large FTP for anyone to visit and download. However, this FTP looks like a library without a librarian. It only has value when you know which book you exactly want to read.

A more advanced way is to make computer a librarian. The computer would play similar songs under the same category. A company named Pandora is a good example. They marked different songs by 400 tags. When you select your favorite song and play, it will also play all the other songs under the same tag. But this looks like a one-way road. Human decisions are smart only in terms of limited music archives. If the music archives are enlarged to millions of songs, Pandora’s style would not be that effective, for we cannot say all users have the same appreciation and understanding as the one who pasted tags.

“Hu’s work was to teach computer to ’listen to the music.’ Different from the way humans listen to music, Hu had to convert the physical features of the music into numbers, and the computer would compare and work on them.”

Is there possibility that Google can teach the computer to become the cleverest librarian? On the one hand, it should know the features of different music well. On the other, it should make customized recommendations.

Computers must learn a lot to have the common sense human has. Though it wouldn’t be as accurate and effective as Pandora’s way at first, it has much room for improvement.

Hu’s work was to teach computer to “listen to the music.” Different from the way humans listen to music, Hu had to convert the physical features of the music into numbers, and the computer would compare and work on them.

For instance, it would be easy to recognize the timber difference between drum and flute. If this difference is converted into numbers – suppose drum to be marked by 2046 while flute marked by 1984, computers without ears can easily separate them. Under well-rounded arithmetic, singers with similar tone colors should be marked in similar numbers: Faye Wong, the Cranberries of Ireland, and Bjork of Iceland might only have difference of 0.1 in numbers.

Then computer might draw a boundless map for all the music. Though computers are only comparing a series of numbers, users would find piano, guitar and rock music in organized categories.

The real calculation is much more complicated than this. Hu needed to convert a song back to frequency spectrum, cut it into thousands of segments in units of hundred millisecond, and take out over 100 features. There might be a drumbeat in this unit and a cymbal beat in the next. You draw all the marks of each song on a paper, and when you put two pieces of such papers together, you see some overlapping and marks in similar positions. You can tell that the two pictures are alike, and the two songs are similar.

It was already a very interesting thought, but Hong Feng wanted something better. After his experience in development of Google Finance, he asked Hu Ning whether there was a way for users to make simple clicks and find different answers on the page, like the way stock search went.

Hong’s thought was very direct viewing for investors: you can get a list of companies of p/e over six, or rather profit margin of over 30 percent. Yet Hu doesn’t agree with this, for music has no p/e or profit margin and users cannot make choices based on that.

But Hong didn’t want to give up this thought easily. He took two weeks in convincing Hu that music, though without clear standard like p/e, would be more fun if there could be some interaction with users.

As a result, two new features were added inito the tight schedule of Google Music’s second edition, including recommendation of similar songs led by Hu Ning, and seletion of songs according to different features.

Lu Yang, doctor of Tsinghua University who joined Google last July, joined Hu in this new direction. The young man who studied signal processing in school didn’t expect to pick up his major when he joined this search engine company. He was required to read dozens of theses by Hu and had to read mathematic books for research. When searching profiles of engineers around the globe, Lin Bin found another engineer in Zurich who had similar academic background with Hu and Lu. Lin invited him to take part in this program.

It wa already a difficulty to find standards applicable to all the songs. People might think that ordinary users could make selections by difference of rhythm, but it was a detour in technical sense.

Lu Yang had experience when he listened to the same song with the Zurich colleague through video conference. The two beat time to the music, and gathered data for about 1,000 songs. The impassible barrio is the multiformity of songs. It’s hard to find the tempo for those songs with light background music. Some songs don’t have the same tempo from beginning to end. People even have different understanding of tempo. The computer had an accuracy of 80 percent in making a distinction of songs in different tempo. Though it is already a hard achievement academically, the chance of one mistake in five songs is not acceptable in practical application. Under these circumstances, Lu suggested that they make distinction from mild to strong, instead of by tempo.

Hu also had difficulty in the similar-song direction. She saw satisfactory performance with her sample product, which covered several thousands of songs, but was frustrated when the range increased to tens of thousands. She had to start over again.

Back in school, she only needed to deal with 100 songs for quantity analysis in order to publish a thesis, but now she was facing hundreds of thousands songs, which was a huge project. Now it was a totally different matter. Chinese language songs and foreign songs of similar timber should be marked differently in numbers. There were also some special cases, e.g. rap music, which was also a big challenge.

Hu Ning and Lu Yang found a simple but effective way to do this huge project. They worked out a solution in small quantity, and made adjustments each time when the quantity was enlarged. Many of the valid code they wrote before had to be abandoned when they moved on to the next stage. It is said that the amount of abandoned code was ten times the one that was left.

The system is relatively stable after processing over 50,000 songs, and they now have 4096 characteristic roots. Somehow it is a product destined for Google – to process 300,000 songs each time takes thousands of computers working tens of hours. Not all companies can do that.

Three weeks before the launch of the revised edition, Li Kaifu and Hong Feng were finally ready to present their work to CEO Eric Schmidt. According to attendees at the presentation, Schmidt, who has rich experience in product development and has a keen eye, commented: “My Chinese colleagues, is there any sharp question for me to ask?”

Indeed, it is a surprising product for Google, no matter in terms of China or the globe.

Will it help Google win back some market shares? It is yet too early to come to this conclusion.

A person close to Google shared his worries with Global Entrepreneur. Given to the bad economic environment this year, Top100 and record companies might not be able to have a good payback as they expect. A more important question is whether Google has the competence to improve and promote the product. In the view of critics, though Google launched quite a few products last year, none of them brought excitement to the market. For instance, Google-Kingsoft Powerword, which is no doubt a good product with great position, quieted down gradually in the past year.

Good news is that everyone of them know there is still much space for improvement, from Li Kaifu to each engineer. According to the plan, after Google Music made its debut, there are still product advertisements, music player and community services to be developed.

Above all, by changing the original competition rules, Google finally filled up its blank and confronts Baidu directly. After three years’ localization, Li Kaifu is now able to sit down, and play himself a song on the internet, with his own product.

When Google Gets Musical by Michael Liang Zhang | Comments (4)

>> More posts

Advertisement