I cut about a third of the text and simplified it a bit.
Version 2.5, notice how I cut out a full explaination of negative and circumstantial evidence.
Getting Your Article on the Front Page: Guidelines for Investigations.
At Bullshido we encourage investigations into questionable claims made by martial artists about their martial arts skills, achievements, and arts.
Principle 1.
Bullshido Investigations will use reliable facts. Facts which are reliable are dependable and trustworthy. Facts are something that actually exists, or are an actual or alleged event or circumstance.
Facts that are offered in the investigation will be relevant, tend to prove or disprove a particular point at issue. For example if I am attempting to prove that David Bannon did not kill a man on the 10th floor of the Byron Hotel in London, the fact that this Hotel only has 6 floors is relevant.
Only with enough reliable facts, can the article writer reach reliable conclusions.
Principle 2:
Certain types of evidence are usually considered reliable, these include:
Legal documents - Arrest warrants and such (see Juko-kai case, Bannon case)
Journalistic sources - Newspaper accounts of events
Video evidence - See Yellow Bamboo and other “ki experts”
Firsthand testimony by recognized expert in martial arts or topical area - Asia, Omega or others.
Pictorial evidence accompanied by substantiating textual evidence. See JFS investigation.
Websites under the control of the person being investigated. (Good for establishing what the person is actually claiming.)
Obviously the reliability of different types of evidence will change from one investigation to another. For example videotape can show a clear picture or be shot in poor lighting, and be poorly focused. Court documents can be informative or baseless and self serving. Some newspaper articles are reliable and well written, others are not worth the paper they are printed upon. Each of the above sources will need to have its reliability individually evaluated before being used.
Principle 3.
What are we looking for?
Investigations will:
a) clearly identify the facts and opinions that it is investigating.
b) indicate their source and where they can be located.
c) accurately reproduce and summarize them for the reader.
d) Sylistically, the writer will also inform the reader early on what they are seeking to prove or disprove, and why this is important.
Principle 4.
To avoid overstating the evidence:
a) Consider all evidence, even conflicting facts.
b) Keep the article fair and cover all aspects of the truth within reason.
Principle 5.
There are several different types of evidence that can be used in an investigation.
a) Direct Evidence is based on personal knowledge or observation and that, if true, proves a fact without inference or presumption. “I saw Master Lout, charge students $500 a piece to learn to levitate.” This evidence is often called testimonal evidence.
b) Documentary Evidence is evidence supplied by a writing or other document which should be traceable to its source so it can be proven that it is real. “I obtained a copy of this court document from the Federal Court at . . .”
c) Negative Evidence is evidence suggesting that an alleged fact does not exist, such as a witness testifying that he or she did not see an event occur.
d) Circumstantial Evidence is based on inference and not on personal knowledge or observation. Sometimes circumstantial evidence can be extremely strong.
Principle 6.
Testimonal or Direct evidence is extremely powerful, but has the limitations of all eyewitness testimony. Eyewitnesses have been known to be biased or simply wrong about what they have seen. Before using eyewitness testimony the writer should attempt to find out if the witness has any biases or hard feelings towards the person being investigated. Such feelings to not automatically or usually invalidate the witnesses’s testimony, but they are important to determine whether the witness is credible.
Similarly the witness may be completely honest but unable to provide reliable information. For example a white belt leaves a McDojo and reports his teacher performed Kata poorly. Perhaps the student is correct and the teacher could not even lift his foot above knee level, but the writer should be careful that he does not repeat conclusions reached by the student about the mechanics of the art that he simply does not have the background to reach.
Testimony from unnamed sources is always inferior to testimony from people willing to provide their real names. While bullshido will sometimes use anonymous testimony, it will be in support of documentary evidence, testimonal evidence from named people, expert opinion, Newspaper accounts of events, Video evidence, etc. Bullshido will try to limit its use of anonymous sources.
Principle 7.
Steve write something about the use of scientific methods and what sort of claims that they can be used to evaluate.
Principle 8.
Using reliable evidence, our goal is to produce investigations with conclusions that are highly probable or reasonably certain to be true.
Note: Many of the definitions used below were borrowed from Black’s Law Dictionary 7th Edition, (St Paul, Minn: West Publishing, 1999).
We are looking for competently researched investigations.
This is the internet. We need well reasoned arguments.
If we don’t have guidelines people will miss some opportunities to make their investigations more reliable
Investigations? This isn’t a detective agency. Consider the pieces more like well done editorials and articles. We don’t have the resources to investigate every issue AND turn out content at a reasonable pace. Look at your favorite blog. Is there info all in the form of a well done investigation? Hell no. If its a decent blog, its a well done opinion piece with information that supports its argument.
secondly they are more likely to turn in a piece of crap which could later cause legal trouble. I.E. making allegations the evidence doesn’t support.
At which point we point out the specific problems, send it back to the writer and await a second draft.
Secondly journalism is not brain surgery
It kinda is. Readibility, an oft overlooked issue in this orgy of scientifical lawyeristic crap is key. Not sounding like a fake ass lawyer doing a science experiment for the New York Times should be a priority here.
if we can get people to follow some of its conventions it will make our articles on Bullshido.com stronger.
Or dry, boring, and nonexistent.
…
(= Just popping in to say I have read this thread, approve of it, and Omega is a pussy.)
The two are not mutually exclusive.
Ooh, doing some research is somehow going to hurt bullshido!!! Did I, or anyone else ever demand that every possible issue be investigated? No, your seeing a false dicotomy. And yes when I read David Corn’s blog, yes occasionally he picks up a phone and calls someone to check a story out, its investigation lite but its there. I think your seeing problems that aren’t there.
Part of successful quality control is to prevent the error before it happens.
And to quote Ronald Reagan, “there you go again.”
There are 2 things that I am concerned about with this standards issue:
- Having credibility
- Not getting sued. (Or being able to defend if we do)
Principle 7.
Steve write something about the use of scientific methods and what sort of claims that they can be used to evaluate.
The conclusions based on the quality and type of evidence used in investigations of Bullshidoka (practitioners of Bullshido) is similar to the conclusions drawn from scientific studies of varying rigor. For instance, in psychology, the only evidence that can indicate causality between one variable and another is a double-blind, longitudinal study with an adequate subject pool. Without delving too much into psychological/statistical principles, the fact is that conclusions drawn from studies are only as valid and reliable as the methods used in the study.
If a study has all men, for example, the results cannot necessarily be applied to women. If a certain result is found in a youth population, it cannot be generalized to a geriatric one. If the study involved only 10 people, the results cannot necessarily apply to the population at large. If the means of measuring a certain variable or concept is inaccurate or unreliable, so will any conclusion derived from it. For example, if I said “I decided how much people were in love by how often they talked politics with each other”, most would reply “What the fuck does POLITICS have to do with love?”
Likewise, if a Bullshido investigation declared “Mr. Smith teaches Bullshido because his a student of his got knocked unconscious by a biker gang,” most readers would say “WTF? How does getting knocked out by a biker gang show martial ability?”
Our site’s investigations follow scientific principles in the following ways:
- The methods we use for investigating are available to anyone to learn and use.
- Our evidence is presented to our peers and the public for scrutiny and disagreement.
- There may be a preconceived idea of how an investigation will turn out, but the conclusions are based on the evidence, rather than evidence being based on our conclusions.
- Data or information will never be falsified (see recent Korean stem cell controversy)
- The methods and standards used for investigations are directly relevant to the topic at hand, and have high amounts of reliability (results can be duplicated) and validity (results are relevant to the question).
- In areas where direct evidence is not available, quality discussion and consensus is how a conclusion is arrived at.
- Investigators and reporters will indicate where conclusions are less reliable or evidence less compelling.
In the interest of simplifying the guidelines, I suggest we only go with this:
"Our site’s investigations follow principles similar to those used in science in the following ways:
- The methods we use for investigating are available to anyone to learn and use.
- Our evidence is presented to our peers and the public for scrutiny and disagreement.
- There may be a preconceived idea of how an investigation will turn out, but the conclusions are based on the evidence, rather than evidence being based on our conclusions.
- Data or information will never be falsified (see recent Korean stem cell controversy)
- The methods and standards used for investigations are directly relevant to the topic at hand, and have high amounts of reliability (results can be duplicated) and validity (results are relevant to the question).
- In areas where direct evidence is not available, quality discussion and consensus is how a conclusion is arrived at.
- Investigators and reporters will indicate where conclusions are less reliable or evidence less compelling."
"Our site’s investigations follow principles similar to those used in science in the following ways:
- The methods we use for investigating are available to anyone to learn and use.
- Our evidence is presented to our peers and the public for scrutiny and disagreement.
- There may be a preconceived idea of how an investigation will turn out, but the conclusions are based on the evidence, rather than evidence being based on our conclusions.
- Data or information will never be falsified (see recent Korean stem cell controversy)
- The methods and standards used for investigations are directly relevant to the topic at hand, and have high amounts of reliability (results can be duplicated) and validity (results are relevant to the question).
- In areas where direct evidence is not available, quality discussion and consensus is how a conclusion is arrived at.
- Investigators and reporters will indicate where conclusions are less reliable or evidence less compelling."
Ok, THIS we can work with. I have some proposed changes, but tonight I’m headed to sleep.
Version 3.2, please propose any changes, and whether we should reprint this in the Bullcave or administrative section for further comments, personally I favor the Bullcave.
Getting Your Article on the Front Page: Guidelines for Investigations.
At Bullshido we encourage investigations into questionable claims made by martial artists about their martial arts skills, achievements, and arts.
Principle 1.
Please use reliable facts. Facts which are reliable are dependable and trustworthy. Facts are something that actually exist, or are an actual or alleged event or circumstance.
Facts that are offered in the investigation will be relevant, tend to prove or disprove a particular point at issue. For example if I am attempting to prove that David Bannon did not kill a man on the 10th floor of the Byron Hotel in London, the fact that this Hotel only has 5 floors and a basement is relevant.
Only with enough reliable facts, can the article writer reach reliable conclusions.
Principle 2:
Certain types of evidence are usually considered reliable, these include:
Legal documents - For example arrest warrants or court transcripts.
Journalistic sources - Newspaper accounts of events.
Video evidence - See Yellow Bamboo and other “ki experts”.
Firsthand testimony by recognized expert in martial arts or topical area.
Pictorial evidence accompanied by an explaintion of what is being shown.
Websites under the control of the person being investigated. (Good for establishing what the person is actually claiming.)
Obviously the reliability of different types of evidence will change from one investigation to another. For example videotape can show a clear picture or be shot in poor lighting, and be poorly focused. Court documents can be informative or baseless and self serving. Some newspaper articles are reliable and well written, others are not worth the paper they are printed upon. Each of the above sources will need to have its reliability individually evaluated before being used.
Principle 3.
What are we looking for?
Investigations will:
a) clearly identify the facts and opinions that it is investigating.
b) indicate their source and where they can be located.
c) accurately reproduce and summarize them for the reader.
d) Sylistically, the writer will inform the reader early on what they are seeking to prove or disprove, and why this is important.
Principle 4.
To avoid overstating the evidence:
a) Consider all evidence, even conflicting facts.
b) Keep the article fair and cover all aspects of the truth within reason.
Principle 5.
There are several different types of evidence that can be used in an investigation.
a) Direct Evidence is based on personal knowledge or observation and that, if true, proves a fact without inference or presumption. “I saw Master Lout, charge each student $500 to learn to levitate.” This evidence is also called testimonal evidence.
b) Documentary Evidence is evidence supplied by a writing or other document which should be traceable to its source so it can be proven that it is real. “I obtained a copy of this court document from the Federal Court at . . .”
c) Circumstantial Evidence is based on inference and not on personal knowledge or observation. Sometimes circumstantial evidence can be extremely strong. For example fingerprint evidence is circumstantial evidence because it relies on a print that was recovered from a crime scene rather then a witness seeing the suspect there. I infer from the print that Joe was present.
d) Negative Evidence is evidence suggesting that an fact does not exist, such as a witness testifying that he or she did not see an event occur when they should have. Negative evidence is often considered weaker then other forms of evidence, so be careful to explain why something should have been observed but wasn’t.
Principle 6.
Testimonal or Direct evidence is extremely powerful, but has the limitations of all eyewitness testimony. Eyewitnesses have been known to be biased or simply wrong about what they have seen. Before using eyewitness testimony the writer should attempt to find out if the witness has any biases or hard feelings towards the person being investigated. Such feelings do not automatically or usually invalidate the witnesses’s testimony, but they are important to determine whether the witness is credible.
Similarly the witness may be completely honest but unable to provide reliable information. For example a white belt leaves a McDojo and reports his teacher performed Kata poorly. Perhaps the student is correct and the teacher could not even lift his foot above knee level, but the writer should be careful that he does not repeat conclusions reached by the student about the mechanics of the art that he simply does not have the background to reach.
Testimony from unnamed sources is always inferior to testimony from people willing to provide their real names. While bullshido will sometimes use anonymous testimony, it will attempt to use this evidence in support of documentary evidence, testimonal evidence from named people, expert opinion, Newspaper accounts of events, Video evidence, etc. Whether the eye witness is anonymous or not, please provide as much relevant information about this witness and their relationship to the party in question. This allows the reader to assess the witnesses credibility.
Principle 7.
Our site’s investigations follow principles similar to those used in science in the following ways:
-
The overwhelming majority of methods we use for investigating are available to anyone to learn and use.
-
Our evidence is presented to our peers and the public for scrutiny and disagreement.
-
There may be a prediction of how an investigation will turn out, but the conclusions are based on the evidence, rather than evidence being based on our conclusions.
-
Data or information will never be falsified (see recent Korean stem cell controversy)
-
The methods and standards used for investigations will be directly relevant to the topic at hand, results can be duplicated, and results are relevant to the question at hand.
-
Investigators and reporters will indicate where conclusions are less reliable or evidence is potentially inconclusive.
Principle 8.
Using reliable evidence, our goal is to produce investigations with conclusions that are highly probable or reasonably certain to be true.
Note: Many of the definitions used below were borrowed from Black’s Law Dictionary 7th Edition, (St Paul, Minn: West Publishing, 1999).
Submission Guidelines
-
Please use reliable facts.
-
Certain types of evidence are usually considered reliable, these include:
A. Legal documents
B. Journalistic sources - Newspaper accounts of events.
C. Video evidence
D. Firsthand testimony by recognized expert in martial arts or topical area
E. Pictorial evidence
F. Websites under the control of the person being investigated.
- What are we looking for?
Submissions will:
a) clearly identify the facts and opinions that it is investigating.
b) indicate their source and where they can be located.
c) accurately reproduce and summarize them for the reader.
d) Sylistically, the writer will inform the reader early on what they are seeking to prove or disprove, and why this is important.
Principle 4.
To avoid overstating the evidence:
a) Consider all evidence, even conflicting facts.
b) Keep the article fair and cover all aspects of the truth within reason.
Remove entirely. No one writes like that. Watch any skeptical production. The opposing view point is only presented in light of preempting refuting the obvious counter. Do lawyers do that shit? I don’t want you up on the stand like “Well, its true that my client DID shoot the guy, but…”
- In submissions relying on particular evidence to make the case, there are several types of evidence that may be used:
a) Direct Evidence is based on personal knowledge or observation and that, if true, proves a fact without inference or presumption. “I saw Master Lout, charge each student $500 to learn to levitate.” This evidence is often called testimonal evidence.
b) Documentary Evidence is evidence supplied by a writing or other document which should be traceable to its source so it can be proven that it is real. “I obtained a copy of this court document from the Federal Court at . . .”
c) Negative Evidence is evidence suggesting that an alleged fact does not exist, such as a witness testifying that he or she did not see an event occur when they should have.
d) Circumstantial Evidence is based on inference and not on personal knowledge or observation. Sometimes circumstantial evidence can be extremely strong. For example fingerprint evidence is circumstantial evidence because it relies on a print that was recovered from a crime scene rather then a witness seeing the suspect there. I infer from the print that Joe was present.
Why is this vocab lesson necessary? Remove entirely
-
Should you make use of an eye witness, anonymous or not, please provide as much relevant information as possible about said witness and his relationship to the party in question. This allows the reader to assess his credibility.
-
Our site’s investigations follow principles similar to those used in science in the following ways (this is fucking cheesy. we’re not DOING science):
A. The methods we use for investigating are available to anyone to learn and use.
Remove this. No, they’re not. This will prevent us from getting the info in the future or will get us in trouble should it be gotten in an underhanded manner.
B. Our evidence is presented to our peers and the public for scrutiny and disagreement.
C. There may be a prediction of how an investigation will turn out, but the conclusions are based on the evidence, rather than evidence being based on our conclusions.
D. Data or information will never be falsified
E. The methods and standards used for contribitions will be directly relevant to the topic at hand, results can be duplicated, and results are relevant to the question at hand.
Results can be duplicated? WTF? We’re not doing science experiements for the most part.
F. Contributers will indicate where conclusions are less reliable or evidence is potentially inconclusive.
- Our goal is to meet certain quality standards in line with the above for all featured submissions.
We’re using science, not doing science.
We’re conducting investigations using scientific principles.
As for the issue of “duplicability”, it states that if anyone else did the research or “ran the experiment” (in this case, visited a school, sparred an instructor, etc.), they would get the same conclusion.
We’re using science, not doing science.
Which is kinda my point.
We’re conducting investigations using scientific principles.
Which is stupid as I’m about to explain.
As for the issue of “duplicability”, it states that if anyone else did the research or “ran the experiment” (in this case, visited a school, sparred an instructor, etc.), they would get the same conclusion.
Do you hear yourself? I’ll cite Frankie Fuller as an example. When Lu first met Frankie, I hear back that he’s this gigantic monster thats absolutely crazy, likes to trade hard and uses a TMA kickboxing style. I see the guy at work and he spars like a pussy using a point sparring style. Gets KTFO in a couple min with a broken nose. Utterly different conclusion, presumably because as a person, HE DID TWO DIFFERENT THINGS. That doesn’t even factor in that martial arts isn’t science and its possible to see the same thing, but reach different conclusions.
So Osiris, would the text below be your final version of the guidelines?
Submission Guidelines
-
Please use reliable facts.
-
Certain types of evidence are usually considered reliable, these include:
A. Legal documents
B. Journalistic sources - Newspaper accounts of events.
C. Video evidence
D. Firsthand testimony by recognized expert in martial arts or topical area
E. Pictorial evidence
F. Websites under the control of the person being investigated.
- What are we looking for?
Submissions will:
a) clearly identify the facts and opinions that it is investigating.
b) indicate their source and where they can be located.
c) accurately reproduce and summarize them for the reader.
d) Sylistically, the writer will inform the reader early on what they are seeking to prove or disprove, and why this is important.
-
Should you make use of an eye witness, anonymous or not, please provide as much relevant information as possible about said witness and his relationship to the party in question. This allows the reader to assess his credibility.
-
Our goal is to meet certain quality standards in line with the above for all featured submissions.
In order of objection:
Principle 4: I think a warning about the danger of overstating evidence is needed, a lot of people reach conclusions without considering what the facts would actually support.
Additionally if lawyers are sure the other side is going to bring up negative information with their witness yes they will bring it up with them before hand. “Dr. Jones, are you being paid to testify as an expert witness here today?”
Principle 5 and 6: It’s not mere vocabulary, its an explaination concerning the differences between various types of evidence with warnings about using negative evidence and eyewitness testimony.
Principle 7: I’ll let Steven handle this one though once again I simplified things a bit to allow us to keep an occasional method secret.
Principle 8: I made an attempt to set a standard of reliability. Would you prefer “clear and convincing evidence?”
Thats what I editted in last time? Yes, short and to the point. Less pretentious.
My proposal is that after we debate this a bit longer that we take both versions either to the administrative section or the bullcave and have the members make a selection between the two.
My replies are above.
I see. I’ll respond in a bit.
Principle 4: I think a warning about the danger of overstating evidence is needed, a lot of people reach conclusions without considering what the facts would actually support.
Thats just bad writing. We shouldn’t need to explain to anyone that will be writing for this site not to pull conclusions out of their ass.
Additionally if lawyers are sure the other side is going to bring up negative information with their witness yes they will bring it up with them before hand. “Dr. Jones, are you being paid to testify as an expert witness here today?”
Huh? Presumably you wouldn’t put out negative info on the client, correct?
Principle 5 and 6: It’s not mere vocabulary, its an explaination concerning the differences between various types of evidence with warnings about using negative evidence and eyewitness testimony.
Too long. No one wants to read all that. A simple statement of “Try to use eye witness testimony and negative evidence (defined as so and so) sparingly as blah blah blah.” That gets us directly to the point and is much easier on the reader.
Principle 7: I’ll let Steven handle this one though once again I simplified things a bit to allow us to keep an occasional method secret.
The problem is that its a different field. Its like following the scientific method of art review. Doesn’t work. Look at Steve’s rebuttal above. Just doesn’t fly.
Principle 8: I made an attempt to set a standard of reliability. Would you prefer “clear and convincing evidence?”
Redundant and adds to the length. I simply shortened it. If we keep adding stuff, it’ll look like on of those software license agreements that nobody reads. A quick blurb telling you not to bootleg their shit will get read. The long legal bullshit will not.