00_Redis_Watch_44.NFO
HBase_and_Redis_LINEs_Storage_System_for_Processing _Over_100_Million_Messages_Per_Day.ENGsub.md
| ii ii ii ii | |
| ii ii ii ii | |
| ii ii ii | |
| i ii ii ii ii i ii iii ii iiii ii ii ii | |
| iiiiii iiiiii iiiiiiii ii iiiiii ii ii iiiiiiiiiiiiii iiiiii iiiiiii | |
| iiiiiiiiiiiiii iiiiiiiii ii iiiiii ii i ii iiiiiiii iiii iiiiiii iiiiiiii | |
| iii iii ii iii iii ii ii i iii ii iiiiii iii ii iii iii iii | |
| iii iiiiiiiii ii ii ii iiiii ii iiii ii ii ii ii iii ii ii | |
| ii iiiiiiiii ii ii ii iiiii iiiiiiiiii ii ii ii ii ii ii | |
| ii iii ii iii ii iii iiiiiiiii iii iii ii iii ii ii | |
| ii iiiiiiii iiiiiiiii iiiiiiiiii iiii iii iiiiiiii iii iiiiiii ii ii | |
| ii iiiiii iiiiiiii ii iiiiii ii iii iiiiiiii iii iiiiii ii ii | |
| i ii ii i i ii i i iii i ii ii i i | |
| Redis Watch #44: https://redislabs.com/redis-watch-archive/44 | |
| Subtitles ripped by: 1T4M4R |
HBase and Redis: LINE’s Storage System for Processing Over 100 Million Messages Per Day
Source: video
Author: Shunsuke Nakamura @sunsuk7tp via LINE Corporation
Method: YouTube auto-generated captions, auto-translated
| Mark | Caption |
|---|---|
| 0:00 | First 12 shots at the first spray that I have a layer |
| 0:03 | Although it is about |
| 0:05 | So the team |
| 0:06 | Some members of the server team |
| 0:08 | We in charge |
| 0:10 | Even developers role of gdi of storage servers |
| 0:14 | It has become |
| 0:16 | In particular |
| 0:17 | As it is written here |
| 0:18 | mlhdx |
| 0:20 | Sixth application |
| 0:21 | It is a log collection |
| 0:23 | House of Representatives of the person in charge of each is divided into these three parts is |
| 0:27 | Stomach |
| 0:27 | In cooperation with each other |
| 0:29 | We are making the storage of line |
| 0:32 | The xe specific server |
| 0:34 | There is also infrastructure following us to corresponding |
| 0:37 | We painting |
| 0:38 | Military and only disturbance Areberu |
| 0:41 | You can concentrate on the development of the advanced level related to it |
| 0:46 | But also it is content to speak this time |
| 0:48 | It is as large following |
| 0:50 | First, service after having described the online storage Overview |
| 0:54 | In order to increase the availability of |
| 0:56 | For ideas that are carried out at the storage level |
| 0:58 | It will introduce |
| 0:59 | And I nh But because the charge of a |
| 1:02 | We will talk about details regarding hds |
| 1:06 | Finally, it will be my talk away the future challenges |
| 1:11 | We will introduce an overview of the first come of storage |
| 1:17 | Next year |
| 1:18 | One one of the messages that users send and receive |
| 1:21 | It has been stored in the storage |
| 1:23 | If it says that it is this doing without |
| 1:25 | First of other users |
| 1:27 | It is to realize the exchange of asynchronous messages |
| 1:31 | Also that the recipient of a message is offline |
| 1:34 | To have you |
| 1:35 | Also send themselves |
| 1:36 | Already when you receive state for the message |
| 1:39 | There is also that you are offline |
| 1:42 | It is assumed that said: |
| 1:44 | A certain amount of messages |
| 1:46 | Medical it to continue to store on the server side was required |
| 1:50 | Message Queue only it was this |
| 1:51 | Such the ones in but is enough do as |
| 1:53 | In the hospital |
| 1:54 | The multi-device Ya from the beginning |
| 1:56 | We are assuming the pc environment |
| 1:59 | Each device |
| 2:00 | the data necessary for the line to fighter |
| 2:04 | Stably to keep |
| 2:06 | Storage was necessary |
| 2:08 | Why is that |
| 2:10 | Satee, such as how the storage of such line |
| 2:13 | That |
| 2:14 | It will be referred to you and that it eh? Being sought |
| 2:17 | I threw two |
| 2:19 | Low latency |
| 2:21 | Nante's message service |
| 2:23 | Hayashi is anyway important |
| 2:25 | And with respect to the data Na to output laser specifically |
| 2:28 | Storage level 03 milliseconds sized response |
| 2:31 | Time will come naturally need |
| 2:34 | Also, not only it |
| 2:36 | If Ukere a message delayed about response time is getting worse |
| 2:39 | So as not to |
| 2:40 | Dressed as sufficient scalability to yeah writing 14 |
| 2:44 | It has become a |
| 2:46 | Another is 20 organizations eyes |
| 2:49 | Stop service for Miss product service to the user |
| 2:53 | It is fatal to take |
| 2:55 | It is with reference to the storage There painting That said |
| 2:58 | Toka's maintenance work |
| 3:00 | Kiriyama am I bought a new car |
| 3:02 | Because it is that yeah inevitable |
| 3:04 | This must be avoided the impact of service |
| 3:09 | By extension excess and lightweight horse at the storage level for the |
| 3:13 | Airport tolerance point of view is becoming an important |
| 3:18 | Such a request so |
| 3:19 | After all, it s type in any touch members |
| 3:22 | In order to meet |
| 3:24 | We who use a variety of oss |
| 3:28 | And the service was selected early days |
| 3:31 | While the play list was to hd |
| 3:33 | Currently Yeah, if necessary monod ed and Cassandra also |
| 3:37 | We Rio |
| 3:42 | However, for pointing out of a share buy used to Komon |
| 3:45 | Since Tsu to have things playlists h base |
| 3:48 | It becomes a story that focused on the two |
| 3:52 | Skilde of tax rate Morino in the high speed |
| 3:55 | It also uses as a cache baseball great Within the storage |
| 3:58 | It has been |
| 4:00 | In ethylene service continued other scalable of s-class |
| 4:04 | Service only 's crater in the statistics and so-called stomach not |
| 4:08 | It is also used for projects |
| 4:11 | Right man in the right place at eline these manicurist hbase |
| 4:14 | The combination also |
| 4:16 | We use |
| 4:18 | The prison a little more in this two |
| 4:20 | And more talk |
| 4:23 | First Ladies |
| 4:25 | Pace |
| 4:26 | Even although I'm a 2011's eh it was introduced |
| 4:29 | Since that time still did not have the function of clustering |
| 4:32 | Such withered ladies cluster out their own dead |
| 4:36 | Made by |
| 4:36 | We have been doing until now |
| 4:39 | When we say that this is how kind of mechanism |
| 4:41 | I will first one master |
| 4:43 | Ml at large |
| 4:45 | I will be considered as such as interest rate a person of if scan master |
| 4:49 | And Hazuki in general |
| 4:51 | Mapping of this Char fine |
| 4:55 | Lowering assigned to the server of the Delice employees 33 |
| 4:58 | Now to the meta information of the cluster, such as |
| 5:00 | You will have to tax field |
| 5:03 | And ladies of the client this Hazuki general |
| 5:06 | With reference to knife on which is fitted the |
| 5:08 | Throw a request to the eh of direct release 3rd |
| 5:12 | It is a mechanism that |
| 5:13 | Periodically Falun the also at the same time towards the client Hazuki general |
| 5:17 | It has removed |
| 5:18 | Yeah the cluster of meta information that there is no update |
| 5:21 | Check to have |
| 5:23 | In addition to a separate cluster manager rather'm Meiji |
| 5:26 | If there are things that the daemon |
| 5:28 | This daemon |
| 5:29 | By performing a health check of the UK side |
| 5:32 | If there is a problem |
| 5:33 | And provides the ability to be promoted to institutional example master does its nuclear |
| 5:38 | To you |
| 5:40 | This story uses the Davis cluster out |
| 5:44 | Then I hd |
| 5:45 | Click here for Man |
| 5:46 | It is originally made the assumption cluster |
| 5:49 | It me uses exactly |
| 5:51 | Before this is the same picture |
| 5:54 | Wintering |
| 5:55 | In the same master Kagata ethylene Service |
| 5:58 | Well today's Davis that was meta-information of the cluster |
| 6:01 | In the same way as the cluster |
| 6:02 | If it is brought to the zookeeper |
| 6:04 | The master has a role to manage it |
| 6:08 | Figure of Ejibesu |
| 6:09 | Free or more servers to manage multiple servers |
| 6:13 | This shared electric bass percent or more - |
| 6:15 | I called the |
| 6:18 | Region of hds |
| 6:19 | At a later date data |
| 6:20 | It is stored in redundant more than hd-ps of District flashy |
| 6:25 | So Yue another legion server logical partitioning |
| 6:29 | Use Kashiwa and |
| 6:33 | hbase- obtain that write performance fast as one of the features |
| 6:37 | Do you have what |
| 6:38 | If you say that whether this is how to implement the |
| 6:41 | Real Gyeongsang specific billing real name |
| 6:43 | In the transaction log circle that h6 |
| 6:46 | By only sequential answer |
| 6:48 | It has been achieved |
| 6:54 | Data lines utilizing such belista-hds |
| 6:57 | Roughly divided into There are three types |
| 7:01 | First from the message the message box care |
| 7:04 | is data such as ition |
| 7:07 | When called What's that this operation |
| 7:09 | The state between the line of server and client |
| 7:12 | It was prepared in order to synchronize |
| 7:14 | It is a line protocol |
| 7:16 | Messe InoSo Gino and say whether right now what kind of things |
| 7:19 | Send Message |
| 7:22 | So Toka To-machi icon has added To-machi Maybe you updated |
| 7:26 | One of the event as one that went |
| 7:28 | We have managed in units called 9 in operation |
| 7:32 | If there are these three data Hate nice down to eat such characteristic |
| 7:36 | Tendency elevator is often said about the latest |
| 7:39 | I have |
| 7:41 | The number in the entire gold of data 100 terabytes |
| 7:46 | And to represent these data |
| 7:48 | Elvis is as persistent storage the hds as cache |
| 7:51 | We use |
| 7:54 | The second data |
| 7:56 | The data, such as user and social graph |
| 7:59 | There really record even if about simple user case of 9 |
| 8:03 | Let alone |
| 8:04 | When the performance we have become easier the |
| 8:06 | There is a number of hundreds of records |
| 8:09 | What about a few tens of terabytes in the whole hope the number of data |
| 8:12 | I is |
| 8:13 | It is a gangrene brother |
| 8:14 | The painting random to come to the amount it replaces less February 24 |
| 8:18 | It is a performance of access |
| 8:20 | So as the eco of the main storage and Baylis |
| 8:23 | I have mainly use |
| 8:24 | The same data has been stored it is redundant to hbase |
| 8:29 | Saigo is |
| 8:30 | Studded |
| 8:31 | It was say statistics and log data you are using in the above service |
| 8:35 | Monono we have managed |
| 8:37 | It had yeah |
| 8:38 | About the total amount of good current petabyte |
| 8:42 | It could use as compared to the deployed mainly placed in hpfs |
| 8:45 | Now do is you have to |
| 8:47 | Since indexing required, if necessary |
| 8:49 | In vertical yeah statistics city Bessho there are also available |
| 8:56 | Overview of storage of 9 this is far |
| 8:59 | Could |
| 9:00 | Then the storage to increase the availability of services |
| 9:03 | We will introduce about devising |
| 9:07 | Specifically, we will introduce the three topics |
| 9:10 | First, each of the tuning of the playlist hbase |
| 9:13 | About |
| 9:15 | The two eyes for duplication of hbase cluster |
| 9:19 | Sign was postponed mlthday deferred |
| 9:22 | It is the realization of high availability of reading |
| 9:27 | First, there is the running after the race |
| 9:29 | On the premise of |
| 9:30 | Interest rate will operate in a single-threaded |
| 9:33 | Nevertheless line of service is be covered position is fast enough |
| 9:37 | New |
| 9:38 | If there is something specific that the he was Tsu failure or slow query |
| 9:42 | Its instance overall performance |
| 9:45 | Yeah is pulled to it |
| 9:46 | It will block the application server |
| 9:50 | This must be something in the line side |
| 9:52 | By that |
| 9:53 | Picture and say as if they were the same |
| 9:55 | Even in cluster manager was the other party detection of problem |
| 9:59 | And it is carried out quickly using it followed the notification |
| 10:03 | It is as in |
| 10:05 | Specifically, Perez is a client timer |
| 10:08 | Before Tohmatsu |
| 10:09 | Through general Ki ze |
| 10:11 | By detecting the way it should be a problem |
| 10:13 | The same and direct sales other storage |
| 10:16 | Application server by |
| 10:19 | It does not block |
| 10:22 | And that is written in the second |
| 10:24 | The 9 is it is but this is actively scripting function |
| 10:27 | Basis I am using |
| 10:29 | Korewae |
| 10:30 | So-called stored procedures |
| 10:32 | In to be able to use as a |
| 10:34 | You can cut the baguette favorite |
| 10:36 | Support lightweight transaction to the same period |
| 10:39 | To have |
| 10:42 | An improved because other clients of connection |
| 10:45 | I put it to have to do |
| 10:49 | Current Lewis of market looks like |
| 10:52 | If you are think that first overall in the 30 class |
| 10:55 | Total in four thousand seven hundred seventy-five schools |
| 10:58 | Third This is a ladies' Masters western instance |
| 11:00 | Of |
| 11:01 | Successor is |
| 11:02 | And use the 48 terabytes of memory in total |
| 11:05 | To have |
| 11:07 | I'm using this amount in quite memory storage |
| 11:10 | Travel is not the kana |
| 11:11 | Have thought |
| 11:14 | Also, as I said I just single-threaded |
| 11:17 | In the line for the US bonds take advantage of the multi-core cpu |
| 11:21 | The other ladies instance of purge to one server |
| 11:24 | To have |
| 11:26 | More cluster or more of its |
| 11:29 | To 2 large easy instances |
| 11:32 | Yes |
| 11:33 | Like this |
| 11:35 | It offers a monitoring management tools of its own race |
| 11:38 | I'm also good because it urethane |
| 11:41 | Health state of each eh cluster at this screen |
| 11:44 | Toka's |
| 11:45 | Memory Usage |
| 11:46 | And to blind Make those say putter stroke sales department |
| 11:49 | I can |
| 11:51 | It is this server a little dangerous left the nest from ear Island |
| 11:56 | I want to somehow |
| 11:59 | Thus eye |
| 12:01 | It came out do it I Na |
| 12:02 | Also this page is presented for each ecluster |
| 12:06 | Here, such as a salad on the pages of each cluster |
| 12:09 | You can see that it had changed is located in the server |
| 12:12 | Tooth |
| 12:14 | The Masters when it was water-cooled dohc problems |
| 12:16 | It is also strong that such function to be promoted to |
| 12:19 | Only mortar Poti the button of English Town Mel |
| 12:21 | You can run safely |
| 12:27 | It performs a whole city-based tuning |
| 12:29 | If you are running a cluster with a certain degree of big bow |
| 12:32 | Machine failure on a daily basis in the Toka problem of risk in the collar or more servers |
| 12:37 | It will occur |
| 12:38 | anyway important to thereby reduce the influence of the service |
| 12:41 | It's as you thought |
| 12:43 | Although There are blood transfusion or point |
| 12:45 | The first point |
| 12:46 | It is a timer and coordination between the basic components |
| 12:50 | First & # as I wrote two 39; skeeper and between the vision server |
| 12:54 | Session Large manner |
| 12:56 | This is has been specified in the default 60 seconds |
| 12:59 | Eh I Decrease |
| 13:01 | As a point that must be noted is the vision server 1 |
| 13:04 | Since all races of the process running on java |
| 13:07 | under the influence of agc |
| 13:09 | It is important not to be removed from the cluster |
| 13:13 | Of the barely 9 |
| 13:14 | It will come to need a timer adjustment |
| 13:19 | Ppi server that allegation |
| 13:21 | pc with idea of hbase client and server legion of |
| 13:24 | Large target |
| 13:26 | Especially in the Senate here it is mainly small |
| 13:29 | Yes |
| 13:30 | And I was told that it is something similar in the book earlier race |
| 13:32 | I is |
| 13:33 | The taste to server for a particular problem |
| 13:35 | The api server to order not to block |
| 13:40 | Since the addition of the Nikkei Symposium Hara state that you are still alive |
| 13:43 | Econo |
| 13:44 | 3 times of two bodies set |
| 13:45 | The short a timer so that it does not request drop of depopulation |
| 13:49 | To you |
| 13:50 | If writing had timed out instead |
| 13:53 | If |
| 13:54 | This api server itself |
| 13:56 | Since I think the local 29 |
| 13:58 | We had eco Savannah |
| 14:00 | After it has been good fishing grounds without vision |
| 14:02 | It has come to retry an iron process |
| 14:10 | The next point is the improvement of the time joined Eri |
| 14:14 | When refrigerated server has gone down |
| 14:17 | H Love to be used in recovery - |
| 14:20 | You should be aware of the accumulation of transaction log |
| 14:23 | If this is going to accumulate |
| 14:25 | Yes |
| 14:26 | There is a problem that extends to review recoverytime8000 |
| 14:30 | Why say or would occur this kind of thing |
| 14:33 | Well it is now located on the mechanism of hbase |
| 14:36 | in several of the region Region server has |
| 14:39 | Since one common I will write a double to feel was 162 Itei |
| 14:44 | This Why? Avoid the impact of risk breeding |
| 14:47 | It is |
| 14:48 | It has collateral vegetables write performance by it |
| 14:51 | Is that |
| 14:52 | But eh When it becomes so |
| 14:55 | Write frequency firefly for that particular server |
| 14:58 | The vision |
| 14:59 | If you would discard as Figure |
| 15:01 | And in Yeah natural writing goods not Vision |
| 15:04 | The x-rayh6 against |
| 15:07 | To become more and more of the wrong without being changes thing called |
| 15:10 | There is also a problem |
| 15:12 | This is the cadastral problems of the British Ji head will occur |
| 15:17 | To avoid this |
| 15:18 | Frequency of writing I'd change the British Akira Madeline chan |
| 15:22 | I think it may be that I'll have all your |
| 15:25 | Vision is doing Maria |
| 15:28 | Gold paint narrowing draw summer |
| 15:31 | Wintering |
| 15:34 | It's a pond mound |
| 15:36 | Writing kind |
| 15:38 | Yes |
| 15:40 | It is placed in a specific server or that is adapted to contemporary |
| 15:45 | Then |
| 15:46 | When it was before making the rice table |
| 15:48 | The size of the region of ets |
| 15:50 | As but become much the same if the same |
| 15:53 | That I'll pre-split |
| 15:55 | And as a solo party is after table |
| 15:58 | How such as to adjust the flash size of media star |
| 16:01 | I have |
| 16:03 | Though this is the way to reduce the force h |
| 16:06 | In order to improve another recovery times of the situation |
| 16:09 | hds has to offer |
| 16:11 | The computerization I clipping |
| 16:13 | In a distributed manner if enabled features such as Ritsumeikan |
| 16:16 | You will do a replay of the log |
| 16:22 | The last point |
| 16:23 | It is to keep the local companies of data of hbase |
| 16:28 | It was assigned to the region server |
| 16:30 | If you play around h it was the realization of vision |
| 16:33 | with the same sport by hd-pfu less of client |
| 16:36 | The data throat |
| 16:38 | We will first write a replica |
| 16:40 | So as long as the problem is not happening |
| 16:42 | Although I kept things that this corridor Rithy |
| 16:46 | But if there is a failure and that Lee John would move |
| 16:49 | Lee John of aging force that had kept John server thereon |
| 16:52 | You collapses |
| 16:54 | The way of data on the machine from the street machine b in FIG. |
| 16:57 | Against |
| 16:58 | There is a need to go the data to take the actual data |
| 17:01 | Students enacted network access |
| 17:03 | Of course the response time will be worse |
| 17:07 | In order to'll solve This is hds- |
| 17:09 | Major compaction constant investigation great victory John run the |
| 17:13 | By |
| 17:14 | The compaction for the same copper data frog time |
| 17:17 | Has been the city environment is created |
| 17:20 | You can keep the re-aging force if so |
| 17:23 | Operation of this major compaction |
| 17:26 | Because yeah h another bag brand process to direct service |
| 17:29 | Not, but I that it Tteyuu influence |
| 17:31 | Because the network and the culture period |
| 17:34 | You need a certain amount of monitoring |
| 17:36 | Is a place to Tteyuu what to do with the above this stage |
| 17:39 | hbs If you are operating in and I think the place kana difficult |
| 17:44 | The course is out next season race function of Oto balancer |
| 17:47 | I'd have a thing called |
| 17:49 | At least old version we are using |
| 17:52 | Yeah so collapses are offset this local amount |
| 17:55 | It does not use a service |
| 18:00 | The second of the pain is the redundancy of hds cluster |
| 18:04 | Data of the British line 1 race cluster two or two cotton |
| 18:09 | Dense cluster |
| 18:10 | It is stored is redundant |
| 18:13 | This is Why? |
| 18:14 | First |
| 18:15 | If you are using fray version |
| 18:17 | Since the name of flashy mail order of the name is a single fault shop |
| 18:20 | It is one of the big merger |
| 18:23 | Also, not just it |
| 18:24 | House |
| 18:25 | A single server or multiple units is |
| 18:28 | And one side with the other cluster when you've boys |
| 18:32 | You can be |
| 18:34 | The third one is quite difficult problem is a lot |
| 18:38 | Yeah hds- |
| 18:40 | By some of the bugs in the font and plant |
| 18:43 | When hbs of operator does not correspond |
| 18:45 | There is a problem that yeah not be automatic control field |
| 18:49 | For example, |
| 18:50 | Yeah particular server is no longer |
| 18:52 | Service to become a state that can not be |
| 18:54 | Yeah carry always and wearing sent to the continued general |
| 18:58 | There is a problem that it is not removed from the cluster |
| 19:01 | When I say do this Toka removed by rain that personnel |
| 19:05 | It was forever to be not that it came to me to Balanchine |
| 19:07 | Also because the problem would have been to continue I |
| 19:10 | Hey the arrangement is duplicated eh cluster |
| 19:13 | By keeping in addition |
| 19:15 | You can have as much as possible to reduce the influence of the yeah the service |
| 19:19 | At the end |
| 19:20 | I wish I have got persists in front of the consideration is in the yeah |
| 19:24 | Even with the destruction of command |
| 19:26 | As not disrupt the line |
| 19:28 | To have |
| 19:32 | How are you doing the duplication of the actual h based cluster |
| 19:35 | If you say |
| 19:36 | Yeah I am simply writing to the client Everest first |
| 19:40 | Especially that update |
| 19:41 | If beech data that looked no luck |
| 19:43 | To accept the large, multiply to cool Ya either one |
| 19:47 | The other is done by aircraft to go without waiting for the picture response |
| 19:50 | You |
| 19:52 | In it appointed both the case where reading is performed in |
| 19:55 | It is a mechanism, such as turn changing the direction which has tradition |
| 20:01 | And that has been introduced into the third |
| 20:03 | The Ya using hds both showed improvement eaves eve of a festival vigil's availability |
| 20:07 | The thing is I have Tsu |
| 20:08 | This is usually |
| 20:10 | Willis are using as the primary |
| 20:12 | Against the data, such as user and social graph |
| 20:15 | Three animals of a particular service |
| 20:17 | If you've a measurement timer of request |
| 20:20 | In both lister city famous water asynchronously |
| 20:22 | How is that return once circle before throwing a get request |
| 20:27 | It is |
| 20:32 | To introduce devised to increase the storage availability of the above |
| 20:35 | It came |
| 20:36 | Then also for more information on the development and operation with respect to hbase |
| 20:41 | I want to release a little |
| 20:44 | Although it is about the environment first come of hbase |
| 20:47 | We are using the version of the tag in the hospital |
| 20:50 | Padang surface was introduced during the 2 strikeouts |
| 20:53 | It is quite old version that closed 90 |
| 20:56 | To the back from the version of some of the top talking |
| 20:59 | And then it is not available |
| 21:01 | Even 9498 regarding the storage, introduced later |
| 21:05 | We use |
| 21:07 | Latest 1.0 is still a place such as verification stage |
| 21:12 | 92 in is amazing and instability has become a partnership in each itself |
| 21:15 | Since I wanted was |
| 21:16 | It makes use of the things that my own development |
| 21:19 | When this is referred to as what kind of things |
| 21:21 | Yes |
| 21:22 | The total number of replication effect on cluster |
| 21:25 | In order to avoid as much as possible |
| 21:26 | even as an independent in pick-up was a ross process |
| 21:29 | It is that to move things |
| 21:32 | Also in partnership 3 itself |
| 21:33 | Transaction log that hrab you are able to talk even the British several times |
| 21:37 | It was the base and |
| 21:38 | It is a mechanism |
| 21:40 | The third of the growing |
| 21:42 | for hbase |
| 21:43 | Use possibly other various storage |
| 21:45 | I'm that |
| 21:46 | You can use a wide range of waiting are ds is in place district court |
| 21:49 | To tell whether large victory |
| 21:51 | Migration of table Toka |
| 21:53 | Repair Toka of lacquerware livelihood between storage |
| 21:56 | Storage Oyobi analysis and things like statistics |
| 21:59 | We use the 2 ema Ttori s |
| 22:02 | After the end that's recently's latest |
| 22:05 | You can make easy the development environment hallway of h base by using a somewhere |
| 22:09 | It has become such a |
| 22:11 | In the development and testing locally using this |
| 22:14 | You can be simplified through hds class and poisoning field |
| 22:21 | way of the Kokodee Zu-based camera shake schema settings |
| 22:24 | We will introduce about |
| 22:27 | Since it is a first basic position it is in is a general law, but |
| 22:31 | When viewed from the Kurainto |
| 22:33 | Yes |
| 22:33 | The time stamp beams column family from wax unfamiliar |
| 22:36 | Tsu was |
| 22:37 | The composed of one element |
| 22:38 | Multidimensional also is a map on the assumption |
| 22:41 | It is stored and is sorted in the order of Sumikyoosu 8 |
| 22:45 | But when viewed in storage internally |
| 22:47 | Large first four |
| 22:49 | Barium-based Ya simple key that intends motorcycle |
| 22:53 | It is stored |
| 22:55 | Also two of the power failure as a feature of edgebase - against six |
| 22:59 | Beauteous and I support the transaction |
| 23:03 | It did on assuming said said data model |
| 23:05 | The summary of the points that we are focused at the time of the s amount |
| 23:09 | I had |
| 23:10 | It is 4. |
| 23:12 | One of the table one first number in Na access time came out |
| 23:16 | It is important to fit the data to the One of the furnace |
| 23:19 | This is its a transaction that was said earlier in special |
| 23:21 | There is also the fact that I |
| 23:23 | Once it kicked enough to become something I think other to also be lower |
| 23:27 | There is also a lot of advantages |
| 23:28 | The fact that the first later become no management of the Region |
| 23:32 | The fact that seawater of a pc is to enter |
| 23:35 | point to such hbase inside of the cache can be effectively utilized |
| 23:38 | I have |
| 23:40 | The two are is to take advantage of the time stamp of hbase |
| 23:45 | All write hbase asynchronous also firmly came in 9th |
| 23:50 | Nde contains and the or |
| 23:51 | Specify the same time stamp to its respective |
| 23:55 | When you run |
| 23:56 | Result the same |
| 23:58 | Value |
| 23:59 | It will be more than or store |
| 24:01 | This allows you to keep the so-called result integrity |
| 24:06 | The third is what to to obtain a scan and Hiruta |
| 24:11 | When you get a plurality of small burr from hds is |
| 24:14 | Than the moon ten |
| 24:15 | well better utilize ets neatly applicator |
| 24:19 | Scan, such as to pass through one of a pc Dee efficiency efficiently |
| 24:23 | You should keep the Schema Design |
| 24:27 | Sign is is better to refrain from familiar simple paintings hbase |
| 24:32 | Riveted is of so storage that have been ethylene |
| 24:35 | The total number of Cantor is very high cost |
| 24:38 | Slow about 100 times that of the roughly speaking, the Gangwon Ima-sama of trellis |
| 24:42 | It is |
| 24:44 | So long as the course riding on the tone |
| 24:46 | I fast |
| 24:47 | Others said I'd fast as long as it is in the cache of hbs |
| 24:50 | Since itself utilizes the heat of self-defense vm |
| 24:53 | There is only the day size |
| 24:55 | Not is scalable |
| 24:57 | Basically ladies are counter in line with mean that |
| 25:00 | We use |
| 25:03 | In order to a little more properly explained that said: |
| 25:06 | Large message as an example of the design of this schema |
| 25:10 | We will introduce the box |
| 25:12 | As a requirement of this message box |
| 25:14 | Less like get to many steel tower amount |
| 25:17 | It is a simple requirement that |
| 25:19 | I read it I'm more times to put the message |
| 25:22 | Are you'll be following sap Yomo~tsu time |
| 25:24 | I think that it is okay with image |
| 25:26 | The data of the actual message box |
| 25:29 | its chat message id that the user send and receive |
| 25:32 | I stored separately |
| 25:34 | The meta-information of such Mataie stage to tear shoes charm Exhibition |
| 25:39 | Yeah it has the same as in another chat |
| 25:41 | For such a message box |
| 25:43 | Initially Ebare was thought at the time to our uniform has not yet been |
| 25:47 | Thing |
| 25:48 | Yeah it was like this skin and a half |
| 25:51 | Three tables |
| 25:52 | Use one column family are expressed |
| 25:56 | To full bloom |
| 25:58 | It dare I comes for from |
| 26:01 | It is the first message meta |
| 26:04 | 1 Tsunoro will for each before Ie emergency id and chat id |
| 26:07 | I make yeah |
| 26:09 | The message id towards the message table to lenticel |
| 26:13 | The time stamp |
| 26:14 | Sequence number is stored starting from the chat vomiting 21 |
| 26:17 | It has been |
| 26:19 | Why are opinion differences in the eco timestamp |
| 26:22 | If you say |
| 26:23 | Get face-to-face of Echizen City |
| 26:25 | Another function of |
| 26:26 | Realization not match Teijin's message id using the operation |
| 26:30 | This is because you are |
| 26:32 | Start of sequence damper and trends |
| 26:34 | The Specify the end sequence of the sequence |
| 26:36 | Get the message id in the range |
| 26:39 | It is interlaced |
| 26:42 | And the second is in a meta downturn |
| 26:44 | The magic I have put the two of Kampa |
| 26:48 | And the last sequence number |
| 26:49 | Annie time |
| 26:50 | It is Kanto |
| 26:52 | Even with things that this last sequence |
| 26:55 | It is used sequencer of the first to have |
| 26:57 | Increments the mind in order to make the message |
| 27:00 | It is a feeling that put its value |
| 27:02 | And it 's only here message restrain |
| 27:06 | When a message enters is incremented hate |
| 27:08 | It is easy that is the Marete cage bottom |
| 27:12 | Sign the b plan |
| 27:14 | Click here for a list of more chat id to join the deposit in the current |
| 27:19 | We have representation |
| 27:20 | Chat id you're put in a column |
| 27:22 | Three years the message box to this near a |
| 27:25 | Although it has been operated as a storage |
| 27:28 | Several found a problem with the month to continue to operate |
| 27:31 | 3. Raise and |
| 27:34 | The first problem first is |
| 27:35 | The time stamp of the message Tsu have is still this |
| 27:38 | Are you've put the sequence |
| 27:40 | It was a problem |
| 27:41 | Since this is not a unix time stamp |
| 27:44 | it is a feature of hbase |
| 27:45 | The old with pet |
| 27:47 | Is a problem in such can not remove the automatic deletion of decode |
| 27:50 | There was |
| 27:52 | So we How and say whether support |
| 27:54 | House |
| 27:55 | The same table size |
| 27:57 | Several and made with of June in May of the body in April of the body table |
| 28:01 | By creating more and more |
| 28:02 | By switching it periodically |
| 28:04 | In great honor work that will remove the shining |
| 28:08 | I had covered the somehow |
| 28:12 | The second problem I saw is about meta of two |
| 28:16 | Earlier it was also to have been like the h base watched say |
| 28:19 | It is slow to answer and even instill of what rice Mari |
| 28:22 | It will heat up of jvm |
| 28:25 | Actually as there were cases is a problem |
| 28:27 | Due to this minute when there is a vigorous chat |
| 28:30 | h But in using the hot spot |
| 28:32 | There was a problem that would make |
| 28:36 | Situation victims problem is had been going this list |
| 28:40 | It is that it is redundant |
| 28:42 | The chat of each user is on the first of two |
| 28:45 | To table |
| 28:46 | Since it is present in gratuities |
| 28:47 | It is possible acquisition if Shiteyare scan this |
| 28:50 | Due there is this table |
| 28:53 | It doubled strength of the data with and regular size increases |
| 28:56 | Also the number of vision dealing in will be doubled |
| 28:59 | Operation of hbase will become a very so twice a |
| 29:03 | In this way there are various problems in the gap of the medulla oblongata rock |
| 29:07 | During the improved ones |
| 29:08 | Here it is anyone who v2 |
| 29:11 | Here to |
| 29:13 | Use two work thing also 3d because did not come out earlier |
| 29:17 | It is represented in a single table |
| 29:21 | First |
| 29:21 | column family named id |
| 29:23 | Applicable to the earlier of the message table in here. |
| 29:26 | message id Contains to |
| 29:29 | Concept be transferred I was there earlier is rather completely if Re here |
| 29:33 | And gone |
| 29:34 | Instead |
| 29:34 | by using the scan after the purchase of hbase |
| 29:37 | The representation Teijin of message id |
| 29:40 | Reblogged |
| 29:42 | In addition to the usual unix time stamp is the time stamp |
| 29:45 | Nde been put |
| 29:46 | Now you'll be crawling automatic deletion |
| 29:49 | 3 mm from another meta to also |
| 29:52 | Here it is became topped even one counter2 there was just |
| 29:56 | Its because cute is |
| 29:58 | Last message ip and the last scene message id |
| 30:02 | Although it called chat another last message id |
| 30:04 | It is a message id you read to the end |
| 30:07 | Using this two |
| 30:08 | Multiple calculations that saw kernel |
| 30:12 | The number of Yes match now is above from the profitability of id in this range |
| 30:17 | By calculating |
| 30:18 | You can calculate an attractive number of messages |
| 30:22 | And in this rather than a little get side |
| 30:24 | The storage side by computing |
| 30:27 | You can markedly to lower |
| 30:30 | This is because the message box - |
| 30:32 | A little better than the packet as it was as a requirement |
| 30:35 | It is because there is less |
| 30:39 | The dialogue that has been standing in value |
| 30:42 | Put towards the timestamp rather than yes Vallier |
| 30:44 | Have you |
| 30:45 | Latest value is large always best by doing so |
| 30:49 | Message id looks like enters |
| 30:53 | This is it introduced a certain skin Zu message box |
| 30:57 | In this way the Estima of design |
| 30:59 | Over with yeah over time |
| 31:02 | And be rewarded doing properly |
| 31:03 | A number of operational problems, such as was earlier v1 |
| 31:07 | In Qian |
| 31:09 | I thought the kana not a Well peace to the birth |
| 31:12 | So |
| 31:14 | 's Table schema important I am Cause I think the |
| 31:20 | We will introduce the actual h base of operation to head |
| 31:25 | First hbase the setup of the cluster using the facility |
| 31:28 | To have |
| 31:29 | the hbase from the beginning to try to step up |
| 31:31 | Then, since there is more than after 20 backer at the discretion |
| 31:34 | I do not want to do |
| 31:35 | Presentation circle by which to some extent use of the facility |
| 31:38 | We long to |
| 31:41 | hbase is up six in the top spot of the component |
| 31:44 | In addition, since multiple versions we are using |
| 31:47 | We have managed to write a recipe each |
| 31:52 | One for every day of the fault correspond. |
| 31:54 | Statistics of the cluster has to some extent automation |
| 31:57 | Toka spot start of the daemon |
| 32:00 | After the Letting John also moved to die and balancing |
| 32:03 | Gastric you |
| 32:03 | More services |
| 32:05 | Rather eh so |
| 32:06 | Vision of management work while some monitoring |
| 32:09 | Since that becomes necessary we just had me a semi-automated |
| 32:13 | Work itself to write a drinking script |
| 32:15 | It is somehow simplified |
| 32:18 | The available tools that can check the status of the h-based cluster |
| 32:21 | It has been |
| 32:23 | It offers something like this |
| 32:25 | Here collectively monitoring of ter from before to obtain the dashboard |
| 32:29 | Possible |
| 32:30 | What is referred to Pena Guo in offset maintenance |
| 32:33 | It is another of the By h in |
| 32:37 | So many maintenance of various types of maps |
| 32:40 | I think that it is a |
| 32:41 | This is not just a simple matter |
| 32:43 | Start-stop Toka Mijon move pit of Ennodo |
| 32:47 | It can also be carried out work |
| 32:51 | In addition to the service scale other than disaster |
| 32:54 | We have to extend the how location-based cluster had fallen |
| 32:56 | We will introduce for the high |
| 33:00 | hbase Since a large number of servers that can easily be of the scaler and face |
| 33:03 | We many times towards the user has runout |
| 33:06 | It enhanced the mackerel |
| 33:07 | And was |
| 33:08 | Kind of cluster also has been increased |
| 33:10 | Because on the one hand it does not increase hbase of engineers in such a momentum |
| 33:14 | There was a problem in the various story |
| 33:17 | First Center of capacity that came out is there was first |
| 33:19 | Problem |
| 33:20 | In a state that it does not put physically and server-side |
| 33:24 | Nde came |
| 33:25 | The time there was a city-based cluster |
| 33:27 | And in over half a year time |
| 33:29 | To solve the whole by migration in the data center |
| 33:32 | Reblogged |
| 33:33 | In the other one it will increase the number of submissions |
| 33:36 | It has been increasing the failed frequency of the hard disk |
| 33:39 | For this purpose, with 12 per 23 Nodoshin |
| 33:42 | Did you have to cope with to stop the hand of development |
| 33:46 | Simply going to the scaler in etan as such are |
| 33:49 | Since the efficiency of the operation has become worse |
| 33:51 | The scale is out that there was at the same time |
| 33:54 | Bus is fall |
| 33:57 | Also was introduced hd situation in the after after it was scaled up |
| 34:00 | Is a corresponding drive for |
| 34:03 | response time and the compaction by iodrive |
| 34:06 | It takes time Toka such things to have been improved |
| 34:09 | There was to be said in its addition to operational level |
| 34:12 | Correspondence of Paris risk failure |
| 34:14 | The machine ever to go to far |
| 34:17 | Was a several times a day has been reduced to about once every few weeks |
| 34:21 | Moreover, this causes other than the drive the love |
| 34:24 | It is due to memory's Toka those sections |
| 34:27 | Also it was dorsoventral to 1/2 of the number of servers |
| 34:31 | Really it's a is called Toko also be done more |
| 34:34 | This time it was becoming bottlenecks |
| 34:36 | vivid size of java Why tax system to hd |
| 34:39 | It has become to the south neck |
| 34:41 | Hds it that can not be reduced any more in the Well is using now |
| 34:45 | It is a version of the problem |
| 34:49 | Although it is the current state of the h-based cluster |
| 34:52 | In political parties total 15 I will not think this class |
| 34:56 | And I am using the node of the 1300 units of iodrive or hd |
| 35:01 | Utilization of risk is all in about 1 petabyte |
| 35:04 | Snappy I have primarily |
| 35:08 | In addition, such as statistics Depp here of a single one |
| 35:12 | clusters is from another * |
| 35:14 | The other twelve sheets each node Ichikawa 6 terabytes is better here in |
| 35:18 | Are moving concentrations disk grew up of |
| 35:20 | Up to the data of 702tb it is operating |
| 35:25 | Disk usage is also here in the 10 petabytes tool position |
| 35:28 | In Snappy Toka astd set |
| 35:30 | It has been Yeah reduction |
| 35:35 | Finally, the introduction about the challenges in the future of the storage level |
| 35:38 | I would like to Ask |
| 35:40 | It's the biggest challenge this year |
| 35:41 | Use the menu in version of its old Furuichi average 90 |
| 35:45 | The |
| 35:45 | 1.0 or more is to raise ver.2 |
| 35:49 | This will have our two sides are many for this |
| 35:53 | There |
| 35:53 | Especially as cotton |
| 35:55 | First of all I will not do it to dramatically recoverytime of legion server |
| 35:59 | This is |
| 36:00 | It came out from earlier Toka's improvement of one form of ahlev |
| 36:04 | And not in the US hds level |
| 36:06 | You now can lead lightweight horse |
| 36:10 | The also rain |
| 36:12 | Well in this version |
| 36:13 | It is that it supports the do to hear |
| 36:17 | Jvma and that has become the country large and attached earlier |
| 36:19 | Heap size |
| 36:21 | No walks |
| 36:21 | Do not you make this by example run more server |
| 36:25 | I thought to Fu that |
| 36:27 | Since last flashy before |
| 36:28 | Name of a moving single thing that not a Kachi-ten |
| 36:31 | But I'm glad |
| 36:35 | Just be directly upgrade to play .91.0 |
| 36:39 | I wanted was |
| 36:39 | And of how to solve this problem and if it is not of course support |
| 36:43 | When the migration of the star level is also from the still |
| 36:46 | It becomes necessary |
| 36:47 | I believe that |
| 36:49 | Also hbase client of not just it |
| 36:52 | Since there is no compatible version |
| 36:54 | You do not have to deal with to this |
| 36:56 | By that |
| 36:57 | This is where you have a variety of pay |
| 37:02 | The other |
| 37:03 | The greater flourished as eca fee |
| 37:06 | After all, multi-data center |
| 37:07 | Matamena storage level of support is important |
| 37:11 | Multiple data centers |
| 37:13 | Keep in place the esd0 o'clock cluster |
| 37:15 | Ieyama night |
| 37:16 | It is that it'll make to possible blood data from the next time registration |
| 37:20 | More contribution of its response purification Toka's improvement of other Tools |
| 37:23 | I want to continue doing it from up to 1.0 |
| 37:28 | The transition in my messaging storage in sulfur |
| 37:31 | This announcement is the end |
| 37:32 | THANK YOU |
Redis, another step on the road
Source: slidedeck
Author: Yi-Feng Tzeng @yftzeng
Method: Google Translate
| Slide # | Source:cn | Translation:en | Context |
|---|---|---|---|
| 46 | 小心 先驅變先烈 |
Heads up Pioneer becomes martyrs |
Server-side sessions with Redis |
| 52 | 偶爾任性是可愛， 一天到晚任性是妖孽。 |
Occasionally wayward cute, All day long wayward evildoer. |
Maximize CPUs usage |
| 57 | 前程四緊： 手頭緊、 眉頭緊、 衣服緊、 時間緊。 |
Occasionally wayward cute, All day long wayward evildoer. |
Memory optimization |
| 69 | 開發都想好自在， 客戶都要靠得住。 |
Developers want a good comfortable, We have reliable customers. |
Availability |
| 85 | 每一個穩定服務背後， 都有一個齷齪的實現。 |
Each service behind a stable, We have a nasty realization. |
Stabilization |
| 106 | 很多事都介於 「不說憋屈」 和 「說了矯情」 之間 |
A lot of things between "Do not say stuck." And "Said a hypocritical" Between |
Low latency |