Month: September 2018

It’s time to stop giving attention to “Ethnicity” and genetic admixture

It’s time to stop giving attention to “Ethnicity” and genetic admixture

[One quick note: As always, we receive no financial benefit or consideration for any product or service we review/recommend/discuss here. Everything we discuss is our opinion alone, and we talk about it because we use it.]

Ancestry has made a lot of noise recently when they updated their Ethnicity estimates, and the now intensified debate about the “accuracy of DNA tests” and the confusion among the general public makes it clear: as a community of serious researchers, we need to be the voice of reason when it comes genetic admixture and call it out for dubiously valuable, largely inaccurate parlor trick that it is. Here’s why:

Ethnicity cannot be tested for. Ever.

Ethnicity is a social construct. Period. If we look at any test, any genealogical tree or other determination it will not build a social link to ones ancestral background. Michael hasn’t been to Ireland, but I have, and despite being able to trace 12.5% of my 3x great grandparents to Ireland, and Ancestry’s admixture pointing to an Irish background, I am not Irish. I visited Ireland as an American…a very obvious American. As will Michael when he visits. Nor will he be mistaken for Beninian when we visit Benin. We are Americans, some with European ancestors some with African ancestors as well, but even with a perfect admixture that could pinpoint our ethnic ancestors exactly…we’re still not German, or Cameroonian, or English/Irish, etc. You can’t test for it, and DNA gives you no indication of how someone identifies ethnically. And that’s important, because Ethnicity is only about how someone identifies themselves and/or how others identify them…it’s not based on a gene. Neither is race, but that’s another rant for another day.

We need to voice a supportable, honest, accurate narrative to drive continued testing…one that will continue after the “Ethnicity” emperor is shown to have no clothes.

It’s not honest

All DNA testing companies, especially 23andMe and Ancestry, are for-profit enterprises that have a strong incentive to grow their number of DNA tests. The larger the test database, the more money the companies charge to sell access to your data. This isn’t to say they are selling personally identifiable data, the data is largely de-identified and aggregated, but it’s YOUR data…and it’s very, very valuable. 23andMe survives almost entirely on the revenue generated from your data, and it’s likely Ancestry is generating a large amount of their revenue from your DNA data as well. And no one’s advertising “come test with us, we are selling to great causes like Michael J. Fox Foundation” [23andMe], they are basing their sales pitch on the shiny bauble that gets the tests in the door: Ethnicity and pretty graphs. The more we play into the Ethnicity debate

It’s not our tool

Ethnicity (as determined by genetic admixture), has almost no genealogical or family history value, and the results will never break a brick wall or significantly add to your family’s stories. First, all of the major providers target who your genetic ancestors were 800-1000 years ago. Even those of us with great trees rarely go back to 1000-1200 AD…and we doubt there would be much value in anyone researching our 28th great grandparents. We have over 1 million 18th GGP’s. Admixture doesn’t rank even among the top 20 tools we use to build our trees, and it doesn’t deliver us any value.

It’s not accurate, and it’s not scientific

16kEthnicityThe biggest red flag from Ancestry’s last update was this: they increased the reference samples from 3,000 tests to 16,000. They have literally spent the last 4 years selling “Ethnicity” to the general public as a great reason to build Ancestry’s test database, even though the entire house of cards was built on 3,000 reference samples. There is no statistically valid data that be gleaned from 3,000 total samples as they relate to our genetic ancestors 1000 years ago. Again, we each had MILLIONS of ancestors 30 generations ago…and to use 3kEthnicity3,000 for all genetic admixture just demonstrates the shoddy science that underpins this process. Even 16,000 is a ridiculously small sample…even if they were each perfectly tied to a region 1000 years ago. “Ethnicity” is just enough science to seem valid enough to be scientific…and just scientific enough to justify the pretty graphs that facilitate the selling of more tests.

It’s hurting genealogy, and it will ultimately turn the public off of genetic DNA testing

Youtube is rife with videos of the general public discussing their “inaccurate” DNA tests, with the testee going into great detail about how they know their Ethnicity and they see something they don’t expect, the test is wrong. There are now new discussions everywhere with people questioning the entire testing process when the “results” can be changed so dramatically by a change by Ancestry. Ancestry is aware of the strain this update is having on the general public, and we can see the efforts they’re making to try and calm people as they go through the update. There are explanations, surveys, etc. to try and make sure the public doesn’t freak out about this change. It’s all just adding more weight to the idea that these tests aren’t accurate/reliable. Since the entire business case for the public taking these tests has been “Ethnicity”, once that’s being exposed as the subjective “art” that it is, the only reason for people to test is being questioned. We will hit a tipping point where our relatives are going to think of DNA testing as a “scam” that’s of no value/dangerous, and it’s going to make the process of getting tests that much harder.

So, what can we do? What impact can we have? Honestly, not much…at least not immediately. But, as the people serious about genealogy we can start being the voice of reason and begin to lay out a better justification for why the public should test, even if the focus of the commercial testing companies is only on adding more samples to their databases. If the thought-leaders and respected voices in the communities turn their back on genetic admixture, that will eventually drive the discussion.

To that end, here’s our suggestions:

  • Stop discussing “Ethnicity” as a testable value – Push back on this basic premise and start to educate the public on why DNA tests have no value as it relates to how they identify ethnically.
  • Don’t give genetic admixture a place at the table – We should no more engage in admixture as a point of genealogical value as we phrenology. They both sound scientific, and their proponents would like them to be seen as science, but neither are science. Even making an anti-admixture discussion elevates it to a “con” in a pro vs. con debate. We need to stop engaging in a debate of equal positions with admixture.
  • Develop other reasons the general public, and our relatives, should submit tests – The tens of millions of tests in various databases have a HUGE value to the genealogical community, and we all benefit as more tests are added. We need to voice a supportable, honest, accurate narrative to drive continued testing…one that will continue after the “Ethnicity” emperor is shown to have no clothes.
  • Be honest with our relatives as they test and help them, and the general public, understand how these tests play into the for-profit world – Those who take tests aren’t purchasing a product, they are the product. 23andMe and Ancestry needs those tests to make a profit, and it’s the only reason why they offer these tests. Let’s discuss that, and what we get in return, to level set everyone’s expectations. If we don’t set these expectations, some scandal will do it for us, and when negative public opinion sets in, we likely will lose the value of having non-experts testing.

Bottom line is that we can see how the reality of DNA testing doesn’t match the perception of the testing public, and all eggs are in the “Ethnicity” basket. As that basket starts to fray, we can either be a part of the rational message that keeps this testing world moving forward, or we can be reactive and wish we could go back to the “good old days” when people were testing without us having to fight for each one.

Ancestry.com takes another step away from its genealogical roots…

Ancestry.com takes another step away from its genealogical roots…

We could see it coming…back in March of 2017, one of our first blog posts was about Ancestry.com’s new tool “We’re Related” (We’re Related app is a lot less frivolous than it first appears). It was a bit of a “hot take” about how it was less silly than it seemed and how it could be very powerful if it’s expanded to a tool that is predictive of your matches.

We’re Related is making suppositions based (apparently) on an algorithm that can draw the line between what you know, and what it guesses is true, to build a potential line for you. If this technology is ever leveraged against some of my brick walls instead a gimmick like linking me to Blake Shelton, Ancestry might really be on to something.

Before we take any victory laps…we have to admit, we were incredibly naive. We never guessed that Ancestry would take this powerful technology and use it to take it’s worst, most frustrating feature, and make it much more dangerous.

The new feature is the “Potential Father/Mother” suggestion, and I’m going to let Carolynn ni Lochlainn detail all the challenges of this new tool, and the risks, in her SPOT ON “From Paper to People” Podcast #27 (From Paper to People: What I Hate About New). Please listen, but her upshot is that this feature is an easy way for those new to genealogy to quickly build out their trees, and the tool forces you to create the ancestor without any sources attached.

One of the biggest drawbacks of Ancestry is the Public Trees that are so often inaccurate, and are often built solely on other people’s unsourced trees. Now, it’s a certainty that these trees are going to start to mushroom, and by design have NO citations attached to the new ancestor.

Ever wonder why Ancestry has delivered even more accurate admixture and even prettier graphs, but none of the tools needed to do serious genealogical research? It’s because there’s no additional revenue from genealogical tools.

The good news is that we as serious users can avoid the downfalls, and use the predictive part of this feature to do the research for us, but we must immediately attach the citations to any newly added ancestor. We, as a community, can also make sure we NEVER use a Member Tree to support a fact. You can link the Member Tree ancestor to yours, but make sure all facts are unselected before you link them. They will see your additional work, and you them, but you will not perpetuate their unsourced facts.

This slideshow requires JavaScript.

But, Ancestry.com isn’t packed full of serious hobbyists/professionals and “Potential Parent” is going to take the problem of Member Trees and make it explode it beyond what we could have imagined. At some point, the tree feature in Ancestry is going to be unusable. Ancestry.com will continue to be a great source of primary research, but it will be nothing more than a data repository for those of us who are serious about this work.

And, back to our naivety…the most frustrating thing is that we should have known better. Again, going back to our vaults, we saw right away that AncestryDNA is here to support genealogy ONLY because it’s a good way to gather DNA tests (Dancing with the Devil: The Tradeoffs of Modern Genealogical Research). Once Ancestry realized that pushing pretty graphs and “ethnicity” was the best way to sell more tests, they pivoted and met their true goal with these tests: the largest DNA database that will generate a tremendous amount of revenue from drug companies, etc. who can leverage your tests to understand how their drugs might work. Ancestry now (or soon will) make more money from monetizing your DNA than it does from supporting our genealogical work.

Screen Shot 2018-09-16 at 9.28.32 AM
How did the public records “Reclaim the Records” paid to get show up here, for paid members only?

Ever wonder why Ancestry has delivered even more accurate admixture and even prettier graphs, but none of the tools needed to do serious genealogical research? It’s because there’s no additional revenue from genealogical tools, but putting more effort into the graphs will drive more people to test, which will grow the database, and grow the revenue stream.

As a community we have to get ready to accept that Ancestry is not a partner in our work, and is not in business to support us or our needs. They exist to generate revenue, and as long as that interest and ours intersect, we’re good, but as they make more money from other streams they are going to sacrifice our needs to focus on revenue. You’re already seeing that with things like “Potential Parents”, more admixture, and their new collections consisting of public records gathered at great expense by groups like Reclaim The Records and putting them behind the paywall.

The genealogy features of Ancestry are still there, for now, but the bad Member Trees we suffer through today are likely going to be remembered as the golden age of online genealogy research.

 

Taking a step back from building our Family Tree

Taking a step back from building our Family Tree

There’s a horrible truth in life that there are only so many hours in the day. We only have so many hours to do what we need to before we start it over again…and as AWESOME it would be to get a 30 hour day, we’ve settled on one with only 24 hours.

With that, the two non-negotiable items in life are a reasonable amount of sleep (6-7 hours) and work. We need food/housing/cars/$ to supporting our family, and our genealogy habit, so we’ll continue to work. That leaves a few hours each night, and a couple of weekend days to manage a household with 5-7 people (depending on which kids are home from college), manage a wonderful marriage, spend time together as a family, and take part in any hobbies we love.

Something has to give, and we decided it will be building out any new branches, or conducting new research, on our family tree

Of course, Family History is one of those hobbies…and we’ve reached the limit of what we can do in the time that we have. We’re going to have to start paring back. We saw this coming over a year ago, when we went from doing our genealogy to adding DNA results to the mix. It became clear that we could literally do nothing but DNA matches for the next 30 years…so we limited our work there. Then came the first great archive that we received. 1000’s of pages of personal documents and photos that are priceless, and which document both our family and a moment of minor historical value in the late 1910’s: The founding of the Progressive Movement in the Republican Party, leading up to the Republican revolt against House Speaker Joe Cannon and Teddy Roosevelt’s “Bull Moose” run for the Presidency in 1912. We knew right away this had the potential to become the dominant focus of our genealogy, and it worried us (Coming up with a plan to manage our new, huge family history collection)

20180824_122814
Rescuing 2000+ dry glass negatives from 1916-1964 is a labor of love…but it’s labor none-the-less

On top of that we’ve become VERY committed to proper citation/labeling of all of documents, and ensuring our public trees are all well cited. And we started this blog. And then we were given another amazing family archive, with more likely to follow. And finally, we rescued nearly 2000 glass plate negatives from a defunct photo studio that operated in the early-to-mid 1900’s in our hometown of Racine, WI that we need to archive, scan, upload to the public, and get ready for donation to the local historical society.

Part of the issue is our success, in that we’ve found SO much amazing materials over the 5 years of doing this that we literally have a lifetime of documents, photos, letters, newspaper clippings, etc. to catalog. We have a strong tree and LOTS of great DNA matches, etc. Part of the issue is we keep adding new stuff to our plate. It’s like Thanksgiving…we need a bit of everything on the biggest plate we can find, with seconds to boot.

We’ve tried to make compromises to keep doing it all. This blog only posts once a week, when we intended on two. And we’ve tried to keep posts to a 500 word maximum, while shelving the vlog we’d always intended. We’ve been VERY judicious on building out DNA links, and when we do we try and make it also double as a chance to make blog posts (like the “Casting a Wide Net” series). Tuesday night is couples night, Saturday night is date night, Sunday is genealogy day, and we try and balance late nights of family history work with pure family nights the next.

Screen Shot 2018-09-11 at 5.03.28 PM
For now, Michael’s 5x GGM will have to be as far as we go on this line…

But, it’s not working. In the last few weeks we had two “come to Jesus” moments. First, as we were trying to write the follow-up on Emily Ott, we found that we had her complete line documented and supported from before her family arrived in the US to her death. But, we weren’t able to really speak to main question we had (and which came out during our Podcast appearance about her): was she “rescued” by the Morse family from retched conditions and treated like a family member, or was she really a hired hand who was beloved, but also always an employee. We realized that we have literally 100 letters between the elder Morse’s and their daughter from that time, as well as the ledger books showing household expenses that include Emily’s salary. The answer is probably right there waiting for us to find it, and knowing that we can’t really publish the definitive “Emily Ott biography” until we review that information.

Then, second, as you read in the conclusion to our “Casting a Wide Net” series, we chased wild geese until we finally went back to the research we already had on-hand and realized that we had a key ancestor completely wrong. Our work to establish a DNA match on that line failed because we had the wrong GGP’s attached, and if we’d used the sources we had to support the facts we using, we would have caught this mistake 4 years ago. But the audio recordings went un-transcribed, and couldn’t be used to support/refute any of our facts.

So, something has to give, and we decided it will be building out any new branches, or conducting new research, on our family tree. For the next year or two we’re going to limit our scope to the following:

  • Process and catalog all family archival material so that it’s: secured, scanned, shared, and documented to where it can be a source for our trees, and others.
  • Determine how to properly clean dry glass plate negatives, and clean, scan, research, document, share, and archive the 2000 negatives we have in advance of donating the collection to an interested Museum.
  • Install a solution to ensure our archive room is kept at 65°/45% humidity year-round.
  • Ensure each Sources in our main, public, family tree is properly cited, with images where possible, and that each Fact is supported by at least one proper source.
  • Properly transcribe and index all family history interviews, so they can be used as proper Sources.
  • PUBLISH!
  • Write our autobiographies, as well as begin to write out what we know about our family…in preparation of publishing the entire history of our families.
  • Ensure that we’re printing out all electronic sources, so that our paper files are complete copies of our electronic files.
  • Spend a little more time with the family!

We’re sure this is a struggle for all of our fellow family historians/genealogists. And I see a lot of amazing blogs with people doing amazing work, with small children like us, and know they are doing the same juggle. If we didn’t have to work this would be a full-time pursuit we’d get more done, but then we’d be significantly less well off and we’d have to juggle the time we spend on clients with the time we need for our personal research. But, this is how we chose to draw the line…and when we get back to building out our tree, we should be much better positioned to make some breakthroughs, and keep everything tidy as we move forward.

There’s no other way to put it – MacKiev’s Family Tree Maker is garbage

There’s no other way to put it – MacKiev’s Family Tree Maker is garbage

One quick point, as always, we receive no financial benefit or consideration for any product or service we review/recommend/discuss here. Everything we discuss is our opinion alone, and we talk about it because we use it.

It’s been a year since MacKiev’s Family Tree Maker dominated this blog, as they struggled to deliver their first version since they took over from Ancestry, and this is what we’ve learned: MacKiev’s Family Tree Maker is garbage.

It’s strong language, but their product continues to be literally dangerous to your data. FamilySync has been a COMPLETE disaster since the moment it was promised (then delivered 4 months late), with MacKiev choosing a synchronization strategy that creates data corruption. Companies like RootMagic have delivered sync without issue, and without risk. But nearly a year since MacKiev finally delivered their FamilySync, it’s still buggy and unreliable. It’s dangerous to your work. It’s garbage.

We’re starting to think that it would have been better to let Ancestry kill the product off. It would have been a hard year, but at least we wouldn’t have wasted that year hoping that MacKiev could actually create/support software.

First off, let me say that I have a LOT of experience working with software delivery…with both commercial products and deploying/supporting in-house developed software. I’ve been doing it for 25 years. And if this was product was deployed in the large corporate environment I currently manage for a Fortune 50 company, we’d pull it out. At all costs. We’d never support this horrible effort, with so little partnership from the vendor.

And, for you loyal readers, you might be asking why we’re using FTM anyways. Didn’t we give away free copies of RootMagic to readers who’d paid for the FTM upgrade last August when MacKiev couldn’t get their act together? We did. And we still use/love RootsMagic…but…

Screen Shot 2018-08-31 at 10.23.36 PM
“Thank you for stopping your work and spending an hour backing up your data, but our TreeSync is so fragile it’s best you put off more research for another 2 hours or so because we don’t know how to design state-aware data synchronization”

The way RM manages citations just isn’t workable for how we support facts in our trees. For example, we created a custom citation in RootsMagic for the 1900 US Census for Roman and Mary Jones and copied the source to each of the 3 Roman’s facts supported by the citation (Name, Birth, and Residence). When we run a TreeSync with Ancestry, everything went off as expected, but it created 3 separate copies of the same citations…one for each fact. Additionally, there’s no central place to manage/edit/view all sources for a tree, which makes it VERY hard to update citations, etc.

So, we still use RootsMagic for our 60+ speculative trees, but we went back to using FTM for our main, public tree.

Since going back to Family Tree Maker, it’s been one disaster after another. First, there was the months of “Orange” sync status late in 2017 (which we missed, luckily, because we’d kicked them to the curb). About 6 months ago we had to start over and re-download the tree from Ancestry, which destroyed all of our source citations. After two months of work, the database corrupted, and we started a cycle of restoring databases, and getting about a month’s use of Family Tree Maker, and then hitting corruption. We have to restore, and repeat the process.

It really seems that this corruption is happening during FamilySync, and if that’s completely inexcusable. Their sync process HAS to be robust enough to not commit records until the system has no risk of corruption. I’ve worked with data replication since 1997 and every tool has a non-destructive method of committing data, and backing out changes if there’s failure/corruption. Plus, RootsMagic has figured this all out…we sync constantly, and there’s never red/orange/green OR corruption. Just repeated success.

The issue is just with MacKiev.

We’ve figured out how to mitigate the risk of FTM corrupting our data by doing constant syncs (change a record, sync, change another record, sync, etc.) and by reviewing the sync reports each time. For example, the last time we had sync stop working, we noticed that the marriage record for Felice’s side of the family was causing changes in the birth record attached to Rick’s grandmother. By deleting both facts we were able to sync successfully and then re-add the facts, and not have to resort to a restore…but I’m only comfortable doing this because I have 20+ years working with/troubleshooting database issues.

Screen Shot 2018-09-01 at 2.10.45 PM
Five easy steps to have your data corrupted anyways!!

But the real nail in the coffin is how MacKiev has chosen to deal with their corruption issues. Instead of architecting a proper deployment, or fixing their code, or building in better error trapping, they turn it back on the users of their product to protect themselves from Family Tree Maker’s failures. They are insisting that you add the following steps to every FamilySync:

  • Backup your database (32 minutes for our 3700 person tree)
  • Compress your database (6 min.)
  • Wait for Green conditions (0-240 min.)

Recently, we did a day’s worth of tree building and citing/sourcing (6 hours) and then we had to stop our work and take nearly an hour to sync. Then, we waited for 2 hours for MacKiev’s FamilySync  to go back to Green. Then, the database corrupted anyways, and we lost all 6 hours of work and had to repeat it.

Screen Shot 2018-08-17 at 9.40.26 AM
When you see this, you’re screwed…they corrupted your database and you lost everything you did since your last backup!!

If we back up every hour, we risk less data, but we cut our productivity in half (work an hour, back up for an hour)…assuming we don’t have to wait for Green.

In the mean time, I switch over the RootsMagic and work on some speculative side project, with regular FamilySync’s, while FTM is dead in the water waiting to “Go Green”.

As much as I have invested in Family Tree Maker, and as much as it is the tool that really meets my reporting needs the best, We’re starting to think that it would have been better to let Ancestry kill the product off. It would have been a hard year, but at least we wouldn’t have wasted that year hoping that MacKiev could actually create/support software.