recent

Titulo

Data Replication Sucks When..

All most all corporations move data from one database to another or from one location to another for various purposes. There are various products like GoldenGate, Data Guard, Attunity Replicate, Tungsten and others for replicating data. I often hear one product is better than other and I don't want to deny that but what matters in the data replication is the design of the database and the replication strategies.

These are my replication best practices which I learnt the hard way so I and my colleagues can sleep without getting a single page for latencies, locked tables or other replication errors. I don't make any exception to these rules for any customers or clients because this is going to bite us and become a nightmare to support it. 
  1. Limit the number of tables to as low as possible. Don't replicate the whole schema or database using a single replication task. Divide them into many jobs and balance big and small tables in terms of volume and data size. 
  2. Don't replicate CLOB or LOB. Many replication tools either don't support or truncate CLOB/LOB column even though they advertise it.
  3. 1-1 table replication between the source and target table. The replication always should be the 1-1 table from source and target and it can't be from many tables in the source to one table in the target. Some tools support MERGE logic but this is a terrible idea when data volumes grow.
  4. Limit the number of Constraints in Target tables. PK is the only constraints that you need to avoid duplicate. Adding too many Constraints slow down the replication and those constraints check should happen in source and not target table.
  5. Absolutely No DDL Replication just DML Replication. Yes, most tool support this feature but I would disable these features to prevent the target table from being ALTERed, TRUNCATEd, and DROPped. 
  6. The table structure should be the same in both source and target. The source and target table should have the same table structures including data types and the only thing that can be different is the constraint on target tables. 
  7. Not your  ETL or Data Manipulation Tool. Remember, this is your replication tool, not your ETL tool. Even though they allow some kind of data manipulation, we should limit the data manipulation for data conversion from one database to another. The will and can slow down the replication.
  8. Don't Advertise real-time replication. There are no tools for real-time replication and never promise anyone regardless of data volumes. With the best database design and replication best practices, you can achieve close to near-real-time but there are so many other factors like network, database load, bandwidth which can increase latencies which you don't have control.
  9. No Replication within the same database. Yes, I have seen a replication happening from one schema to another schema within the same Oracle database. Similarly, in one database to another database within a SQL Server Instance. Don't set up this type of replication even though replication is supported within a database, use script or other database vendor provided tools. It causes a lot of latency and generates redo log that isn't necessarily causing overhead to the database.
  10. No Merge/UPSERT logic. This is similar to that of rule #3, repeating again to reinforce the practice
  11. Never enable Bi-Directional replication feature. This is the most dangerous replication practices of all and I would tell my client there is no such feature out there. Anyone has enabled bi-directional replication? Please tell us more on the project and how do you support when there is data mismatch issue? I would like to learn more and everyone here would like to know more.
Violating these Eleven Golden Replication rules may suffer into a loss of sleep, latency anxiety, replication PTSD and much, much more. I hope you use them at your own risk to benefit your health, the database health and make no exception to these rules. Oops, I just got a latency alert, I will see you after the latency is caught up.

Interested in working with me? I can be reached at pbaniya04[at]gmail.com for any questions, consulting opportunities or you may drop a line to say HELLO. Thank your again for visiting my blog and looking forward to serving you more.

Have a Database-ious Day!

101 comments

  1. On the off chance that the exhibit isn't numerical, there isn't much we can do with the exception of perusing and composing its information. ExcelR Data Science Courses

    ReplyDelete

  2. This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck.
    data science course in guwahati

    ReplyDelete
  3. This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck.
    Data Science Institute in Bangalore

    ReplyDelete
  4. I am looking for and I love to post a comment that "The content of your post is awesome" Great work!

    Simple Linear Regression

    Correlation vs Covariance

    ReplyDelete
  5. 360DigiTMG GunturJuly 7, 2020 at 1:52 AM

    I feel very grateful that I read this. It is very helpful and very informative and I really learned a lot from it.

    data science course in guntur

    ReplyDelete
  6. I feel very grateful that I read this. It is very helpful and very informative and I really learned a lot from it.
    Data Analytics Course in Pune
    Data Analytics Training in Pune

    ReplyDelete
  7. Very interesting blog. Many blogs I see these days do not really provide anything that attracts others, but believe me the way you interact is literally awesome.You can also check my articles as well.

    Data Science In Banglore With Placements
    Data Science Course In Bangalore
    Data Science Training In Bangalore
    Best Data Science Courses In Bangalore
    Data Science Institute In Bangalore

    Thank you..

    ReplyDelete
  8. Hello, I have browsed most of your posts. This post is probably where I got the most useful information for my research. Thanks for posting, we can see more on this. Are you aware of any other websites on this subject.
    data science training
    data science course
    data science course in hyderabad

    ReplyDelete
  9. This post is great. I reallly admire your post. Your post was awesome.
    data science course in Hyderabad

    ReplyDelete
  10. I feel a lot more people need to read this, very good info!.Learn best Data Science Course in Hyderabad

    ReplyDelete
  11. I am impressed by the information that you have on this blog. Thanks for Sharing
    Ethical Hacking in Bangalore

    ReplyDelete
  12. Very interesting blog. Many blogs I see these days do not really provide anything that attracts others, but believe me the way you interact is literally awesome.You can also check my articles as well.

    Rowe Rowe
    Manager Rowe Rowe
    Rapper Rowe Rowe

    Thank you..

    ReplyDelete
  13. A good blog always comes-up with new and exciting information and while reading I have felt that this blog really has all those quality that qualify a blog to be a one.

    360gigitmg.com

    ReplyDelete
  14. The article unambiguously showed each of the positive and negative sides of the issue. This is indeed a thought infuriating article.
    Data Science training in Mumbai
    Data Science course in Mumbai
    SAP training in Mumbai

    ReplyDelete
  15. Highly appreciable regarding the uniqueness of the content. This perhaps makes the readers feels excited to get stick to the subject. Certainly, the learners would thank the blogger to come up with the innovative content which keeps the readers to be up to date to stand by the competition. Once again nice blog keep it up and keep sharing the content as always.

    360DigiTMG Business Analytics Course

    ReplyDelete
  16. Attend The Data Analyst Course From ExcelR. Practical Data Analyst Course Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Data Analyst Course.
    Data Analyst Course

    ReplyDelete
  17. Amazing Article ! I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
    Simple Linear Regression
    Correlation vs covariance
    data science interview questions
    KNN Algorithm
    Logistic Regression explained

    ReplyDelete
  18. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data sciecne course in hyderabad

    ReplyDelete
  19. This was certainly one of my preferred web journals. Each post distributed impressed me.
    data science courses in delhi

    ReplyDelete
  20. There is no dearth of Data Science course syllabus or resources. Learn the advanced data science course concepts and get your skills upgraded from the pioneers in Data Science.
    data science course bangalore
    data science course syllabus

    ReplyDelete
  21. help in understanding the recent trends and customer demands, also to analyze customer feedbacks and most importantly handle inventory and warehousing. data science course syllabus

    ReplyDelete
  22. Incredibly in general very intriguing post. I was looking for such an information and took pleasure in scrutinizing this one. Keep posting. An obligation of appreciation is all together for sharing.data analytics course in Hyderabad

    ReplyDelete
  23. Attend The Machine Learning Course Bangalore From ExcelR. Practical Machine Learning course Bangalore Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Machine Learning course Bangalore.
    Machine Learning Course Bangalore

    ReplyDelete
  24. Very nice blog and articles. I am realy very happy to visit your blog. Now I am found which I actually want. I check your blog everyday and try to learn something from your blog. Thank you and waiting for your new post.

    data science course in India

    ReplyDelete
  25. You might comment on the order system of the blog. You should chat it's splendid. Your blog audit would swell up your visitors. I was very pleased to find this site.I wanted to thank you for this great read!!
    Artificial Intelligence Course

    ReplyDelete
  26. I would like to thank you for the efforts you have made in writing this article. I am hoping the same best work from you in the future as well. In fact your creative writing abilities has inspired me to start my own Blog Engine blog now. Really the blogging is spreading its wings rapidly. Your write up is a fine example of it.
    machine learning course training in Hyderabad

    ReplyDelete
  27. A great service is that the mistake they make will be send to all other AI computers linked to them so that they will also not make that same mistake. data science course in india

    ReplyDelete
  28. You completed a number of nice points there. I did a search on the issue and found ExcelR Data Analytics Course nearly all people will have the same opinion with your blog.

    ReplyDelete
  29. I like viewing web sites which comprehend the price of delivering the excellent useful resource free of charge. I truly adored reading your posting. Thank you!
    business analytics course

    ReplyDelete
  30. As always your articles do inspire me. Every single detail you have posted was great.
    data science certification in noida

    ReplyDelete
  31. Great post i must say and thanks for the information. Education is definitely a sticky subject. However, is still among the leading topics of our time. I appreciate your post and look forward to more.
    Data Science Course in Bangalore

    ReplyDelete
  32. Happy to visit your blog, I am by all accounts forward to more solid articles and I figure we as a whole wish to thank such huge numbers of good articles, blog to impart to us.
    data scientist course

    ReplyDelete

  33. I see the greatest contents on your blog and I extremely love reading them.
    Best Institute for Data Science in Hyderabad

    ReplyDelete
  34. I am really enjoying reading your well written articles. It looks like you spend a lot of effort and time on your blog. I have bookmarked it and I am looking forward to reading new articles. Keep up the good work.
    artificial intelligence course in pune

    ReplyDelete
  35. Through this post, i do know that your smart information in fiddling with all the items was very useful. I advise that this can be the primary place wherever I notice problems i have been sorting out. you've got a creative nevertheless engaging approach of writing.
    data scientists training

    ReplyDelete
  36. If you are looking for Illinois license plate sticker renewals online, you have to go to the right place. We have the fastest Illinois license plate sticker renewals in the state.
    Data Science Course in Pune

    ReplyDelete
  37. I just got to this amazing site not long ago. I was actually captured with the piece of resources you have got here. Big thumbs up for making such wonderful blog page!
    artificial intelligence course in pune

    ReplyDelete
  38. I need to communicate my deference of your composing aptitude and capacity to make perusers read from the earliest starting point as far as possible. I might want to peruse more up to date presents and on share my musings with you.
    big data analytics malaysia

    ReplyDelete
  39. Thanks for your nice post I really like it and appreciate it. My work is about Custom Vape Cartridge Boxes. If you need perfect quality boxes then you can visit our website.

    ReplyDelete
  40. i am glad to discover this page : i have to thank you for the time i spent on this especially great reading !! i really liked each part and also bookmarked you for new information on your site.
    data science courses in hyderabad

    ReplyDelete
  41. I am really enjoying reading your well written articles. It looks like you spend a lot of effort and time on your blog. I have bookmarked it and I am looking forward to reading new articles. Keep up the good work.
    artificial intelligence course in pune

    ReplyDelete
  42. I’ve been browsing on-line greater than 3 hours as of late, but I by no means found any attention-grabbing article like yours. It’s pretty value sufficient for me. In my view, if all web owners and bloggers made just right content material as you did, the web can be a lot more useful than ever before.
    data scientist training in hyderabad

    ReplyDelete
  43. It was a good experience to read about dangerous punctuation. Informative for everyone looking on the subject.
    data scientist training and placement in hyderabad

    ReplyDelete
  44. I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors.
    data science course in chennai

    ReplyDelete
  45. This website and I conceive this internet site is really informative ! Keep on putting up!
    ai course

    ReplyDelete
  46. I just couldn't leave your website before telling you that I truly enjoyed the top quality info you present to your visitors? Will be back again frequently to check up on new posts.
    data scientist training in hyderabad

    ReplyDelete
  47. I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors.
    data science training in chennai

    ReplyDelete
  48. i am glad to discover this page : i have to thank you for the time i spent on this especially great reading !! i really liked each part and also bookmarked you for new information on your site.
    artificial intelligence training in chennai

    ReplyDelete
  49. Thanks for posting the best information and the blog is very important.artificial intelligence course in hyderabad

    ReplyDelete
  50. I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors.
    data science course

    ReplyDelete
  51. Thanks for bringing such innovative content which truly attracts the readers towards you. Certainly, your blog competes with your co-bloggers to come up with the newly updated info. Finally, kudos to you.

    Data Science Course in Varanasi

    ReplyDelete
  52. I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors.
    business analytics courses

    ReplyDelete
  53. I am sure that this is going to help a lot of individuals. Keep up the good work. It is highly convincing and I enjoyed going through the entire blog.
    data science course

    ReplyDelete
  54. I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors.
    aws training in hyderabad

    ReplyDelete
  55. I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors.
    iot course in bangalore

    ReplyDelete
  56. Extremely overall quite fascinating post. I was searching for this sort of data and delighted in perusing this one. Continue posting. A debt of gratitude is in order for sharing. best institute for cloud computing in hyderabad

    ReplyDelete
  57. Amazing Article! I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.If you are Searching for info click on given link
    Data science course in pune

    ReplyDelete
  58. This article gives the light in which we can observe reality. This is a very nice one and gives in depth information. Thanks for this nice article.
    data scientist course in hyderabad

    ReplyDelete
  59. Amazingly by and large very interesting post. I was looking for such an information and thoroughly enjoyed examining this one. Keep posting. An obligation of appreciation is all together for sharing.business analytics course in gwalior

    ReplyDelete
  60. I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors
    data science course in thiruvananthapuram

    ReplyDelete
  61. Well we really like to visit this site, many useful information we can get here. data science course in mysore

    ReplyDelete
  62. I am the first time visiting this page, really awesome and knowledgeable content. I bookmarked your site for future blogs. Keep up this work. Thank you.
    AI Patasala Data Science Course in Hyderabad

    ReplyDelete
  63. Excellent content ,Thanks for sharing this .,
    Online teaching platform
    Online teaching tool

    Online teaching platform

    ReplyDelete
  64. Extremely overall quite fascinating post. I was searching for this sort of data and delighted in perusing this one. Continue posting. A debt of gratitude is in order for sharing.data science course in kolhapur

    ReplyDelete
  65. Thank you so much for doing the impressive job here, everyone will surely like your post.
    cyber security course in malaysia

    ReplyDelete
  66. Stupendous blog huge applause to the blogger and hoping you to come up with such an extraordinary content in future. Surely, this post will inspire many aspirants who are very keen in gaining the knowledge. Expecting many more contents with lot more curiosity further.

    Data Science Certification in Bhilai

    ReplyDelete
  67. Extraordinary blog went amazed with the content that they have developed in a very descriptive manner. This type of content surely ensures the participants to explore more themselves. Hope you deliver the same near the future as well. Gratitude to the blogger for the efforts.

    Data Science Training

    ReplyDelete
  68. Extremely overall quite fascinating post. I was searching for this sort of data and delighted in perusing this one.
    Continue posting. A debt of gratitude is in order for sharing.
    data science training in gwalior

    ReplyDelete
  69. Extremely overall quite fascinating post. I was searching for this sort of data and delighted in perusing this one.
    Continue posting. A debt of gratitude is in order for sharing.data science course in warangal

    ReplyDelete
  70. I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors
    business analytics course in varanasi

    ReplyDelete
  71. Extremely overall quite fascinating post. I was searching for this sort of data and delighted in perusing this one.
    Continue posting. A debt of gratitude is in order for sharing.
    data scientist course in warangal

    ReplyDelete
  72. Wonderful blog. I delighted in perusing your articles. This is genuinely an incredible perused for me. I have bookmarked it and I am anticipating perusing new articles. Keep doing awesome!.data science course in bhubaneswar

    ReplyDelete
  73. I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors
    data science training in delhi

    ReplyDelete
  74. I would like to thank you for the efforts you have made in writing this article. I am hoping the same best work from you in the future as well. In fact your creative writing abilities has inspired me to start my own Blog Engine blog now. Really the blogging is spreading its wings rapidly. Your write up is a fine example of it. Project Management Course

    ReplyDelete
  75. I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors
    data scientist course in delhi

    ReplyDelete
  76. I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors
    data science course delhi

    ReplyDelete
  77. Amazingly by and large very interesting post. I was looking for such an information and thoroughly enjoyed examining this one.
    Keep posting. An obligation of appreciation is all together for sharing.
    data science course in gwalior

    ReplyDelete
  78. Good to visit your weblog again, it has been months for me. Nicely this article that i've been waiting for so long. I will need this post to total my assignment in the college, and it has the exact same topic together with your write-up. Thanks, good share.
    data analytics course in hyderabad

    ReplyDelete
  79. Extremely overall quite fascinating post. I was searching for this sort of data and delighted in perusing this one.
    Continue posting. A debt of gratitude is in order for sharing.
    data analytics course in kolhapur

    ReplyDelete
  80. This post is very simple to read and appreciate without leaving any details out. Great work! PMP Training in Malaysia

    ReplyDelete
  81. I’m excited to uncover this page. I need to to thank you for ones time for this particularly fantastic read!! I definitely really liked every part of it and i also have you saved to fav to look at new information in your site. PMP Course

    ReplyDelete
  82. Thanks for sharing such useful information with us. I hope you will share some more info about your blog. Please keep sharing. We will also provide HR Software India

    ReplyDelete
  83. Our Data Science certification training with a unique curriculum and methodology helps you to get placed in top-notch companies.

    ReplyDelete
  84. Thanks to all the contents that appear in this guide, those in charge of business management and administration can find out about the characteristics that Microsoft Dynamics 365 presents and the benefits that it brings to the different departments of the companies. Dynamics 365 Customer Service

    ReplyDelete
  85. Eight fundamental steps to use an online ERP
    Organize a suitable team: All software changes are traumatic for companies. This fact is more accentuated when it comes to a large company and complex software.
    If we start with a team that is enthusiastic about change and committed to the new software, it will be much easier to extend that encouragement to the rest of the organization.
    Incorporation by modules: It can also be efficient to incorporate a new online ERP by modules . Implement it only in one department and see the results and then carry out implementations in other departments.
    Good practices: When a new ERP in the cloud is incorporated into business management, it is time to re-imagine what the best operation of the company will be. If you just copy what you already have, you will not get the most out of the new software nor will it be a step forward for the company.
    Dynamics 365 Business Central Implementation
    NAV to Business Central Upgrade

    ReplyDelete
  86. A person who works on data, manipulates it, and takes the right outcomes from it, but did not deliver the message properly, then all the efforts are a waste of time.data science training in kolkata

    ReplyDelete
  87. "If you are also one of them and want to know what the companies demand from the data scientists to do in their organization, you have come to the right place.data science course in kolkata"

    ReplyDelete
  88. Thank you very much for letting me read this. It is incredibly beneficial and instructive, and I gained a lot from it. Data science course in bhopal

    ReplyDelete

Powered by Blogger.