All posts in category scholarly

Make more money

So I enjoy Ramit Sethi’s Netflix series on money management – fundamentally it is about money coming in and money going out and the ability to balance a budget. On occasion I see other budget coaches focus on trivial expenses (the money going out) whereas for me (and I suspect the majority of folks reading this blog with higher degrees and technical backgrounds) you should almost always be focused on finding a higher paying job.

Lets go with a common example people use as unnecessary discretionary spending – getting a $10 drink at Starbucks every day. If you do this, over the course of a 365 day year, you will have spent $3650 additional dollars. If you read my blog about coding and statistics and that expense bothers you, you are probably not making as much money as you should be.

Ramit regularly talks about asking for raises – I am guessing most people reading this blog if you got a raise it would be well over that Starbucks expense. But part of the motivation to write this post is in reference to formerly being a professor. I think many criminal justice (CJ) professors are underemployed, and should consider better paying jobs. I am regularly starting to see public sector jobs in CJ that have substantially better pay than being a professor. This morning was shared a position for an entry level crime analyst at the Reno Police Department with pay range from $84,000 to $102,000:

The low end of that starting pay range is competitive with the majority of starting assistant professor salaries in CJ. You can go check out what the CJ professors at Reno make (which is pretty par for the course for CJ departments in the US) in comparison. If I had stayed as a CJ professor, even with moving from Dallas to other universities and trying to negotiate raises, I would be lucky to be making over $100k at this point in time. Again, that Reno position is an entry level crime analyst – asking for a BA + 2 years of experience or a Masters degree.

Private sector data science jobs in comparison, in DFW area in 2019 entry level were often starting at $105k salary (based on personal experience). You can check out BLS data to examine average salaries in data science if you want to look at your particular metro area (it is good to see the total number in that category in an area as well).

While academic CJ salaries can sometimes be very high (over $200k), these are quite rare. There are a few things going against professor jobs, and CJ ones in particular, that depress CJ professor wages overall. Social scientists in general make less than STEM fields, and CJ departments are almost entirely in state schools that tend to have wage compression. Getting an offer at Harvard or Duke is probably not in the cards if you have a CJ degree.

In addition to this, with the increase in the number of PhDs being granted, competition is stiff. There are many qualified PhDs, making it very difficult to negotiate your salary as an early career professor – the university could hire 5 people who are just as qualified in your stead who aren’t asking for that raise.

So even if you are lucky enough to have negotiating power to ask for a raise as a CJ professor (which most people don’t have), you often could make more money by getting a public sector CJ job anyway. If you have quant skills, you can definitely make more money in the private sector.

At this point, most people go back to the idea that being a professor is the ultimate job in terms of freedom. Yes, you can pursue whatever research line you want, but you still need to teach courses, supervise students, and occasionally do service to the university. These responsibilities all by themselves are a job (the entry level crime analyst at Reno will work less overall than the assistant professor who needs to hustle to make tenure).

To me the trade off in freedom is worth it because you get to work directly with individuals who actually care what you do – you lose freedom because you need to make things within the constraints of the real world that real people will use. To me being able to work directly on real problems and implement my work in real life is a positive, not a negative.

Final point to make in this blog, because of the stiff competition for professor positions, I often see people suggesting there are too many PhDs. I don’t think this is the case though, you can apply the skills you learned in getting your CJ PhD to those public and private sector jobs. I think CJ PhD programs just need small tweaks to better prepare students for those roles, in addition to just letting people know different types of positions are available.

It is pretty much at the point that alt-academic jobs are better careers than the majority of CJ academic professor positions. If you had the choice to be an assistant professor in CJ at University of Nevada Reno, or be a crime analyst at Reno PD, the crime analyst is the better choice.

4 Comments
by Andy Wheeler on June 22, 2023  •  Permalink
Posted in Crime Analysis, Criminal Justice, data science, scholarly
Tagged

Posted by Andy Wheeler on June 22, 2023

https://andrewpwheeler.com/2023/06/22/make-more-money/

Criminology not on the brink

I enjoy reading Jukka Savolainen’s hot takes, most recently Give Criminology a Chance: Notes from a discipline on the brink. I think Jukka is wrong on a few points, but if you are a criminologist who goes to ASC conferences, definitely go and read it! To be specific, in addition to the title here are two penultimate paragraphs in full that I mostly disagree with:

I arrived in Atlanta with a pessimistic view of academic criminology. During my 30 years in the field, the scholarship has become increasingly political and intolerant of evidence that contradicts the progressive narrative. The past few years have been particularly discouraging for those who care about scientific rigor and truth. Despite these reservations, I approached the ASC meeting with an open mind.

The situation is far from hopeless. True, criminology possesses precious little viewpoint diversity. Much of the scholarship is more interested in pursuing a political agenda than objective truth. The ASC’s outward stance as a politically neutral arbiter of scientific evidence is at odds with its recent history as an activist organization.

Although his take on a generic American Society of Criminology experience is again not misleading and accurate, I am not so sure about the assessment of the trend over time, e.g. “increasingly political and intolerant”. Nor do I think criminology has too “little viewpoint diversity”.

The latter statement is to be frank absurd. For those who haven’t been to an ASC conference, there are no restrictions to who can become a member of the American Society of Criminology. The yearly conference is essentially open as well – you have to submit an abstract for review, but I have never heard of an abstract being turned down (let me know if you are aware of an example!) So you really get the kaleidoscope (as Jukka articulated). Policing scholars, abolitionists, quantitative, qualitative, ghost criminology – criminologists are a heterogeneous bunch.

About the only way to steelman the statement “precious little viewpoint diversity” is to say something more like certain opinions in the field are rewarded/punished, such as being in advanced positions at ASC, or limiting what gets published in the ASC journals (Criminology or Criminology and Public Policy). Or maybe that the average mix of the field slants one way or another (say between pro criminal justice or critical criminal justice).

I have not been around 30 years like Jukka, and I suppose I lost my card carrying criminologist privileges when I went to the private sector, but I haven’t seen any clear change in the nature of field, the ASC conference, or what has been published, in the last ~10 years I have been in a reasonable position to make that judgment. I think Jukka (or anyone) would be hard pressed to quantify his perception – but certainly open to real evidence if I am wrong here (again just my opinion based on fewer years of experience than Jukka).

As a side story, I have heard many of my friends who do work in policing state that they have been criticized for that by colleagues, and subsequently argue our field is “biased against cops”. I don’t doubt my friends personal experiences, but I have personally never been criticized for working with the police. I have been criticized by fellow policing scholars as “downloading datasets” and “not being a real policing scholar”. I know qualitative criminologists who think they are biased against in the field (based on rates of qualitative publishing). I know quantitative criminologists who have given examples of bias in the field against more rigorous empirical methods. I know Europeans who think the field is biased towards Americans. I bet the ghost criminologists think the living are biased against the (un)dead.

I think saying “Much of the scholarship is more interested in pursuing a political agenda than objective truth” is a tinge strong, but sure it happens (I am not quite sure how to map “much” to a numeric value so the statement can be confirmed or refuted). I would say being critical of some work, but then uncritically sharing equally unrigorous work that confirms your pre-conceived notions is an example of this! So if you think one or more is “much”, then I guess I don’t disagree with Jukka here – to be clear though I think the majority of criminologists I have met are interested in pursuing the truth (even if I disagree with the methods they use).

So onto the last sentence of Jukka’s I disagree with, “The ASC’s outward stance as a politically neutral arbiter of scientific evidence is at odds with its recent history as an activist organization.”. But I disagree with this because I personally have a non-normative take on science – I don’t think science is wholly defined by being a neutral arbiter of truth, and doing science in the real world literally involves things that are “activist”.

I believe if you asked most people with Phds what defines science, they would say that science is defined via the scientific method. I personally think that is wrong though. I think about the only thing we share as scientists are being critique-y assholes. The way I do my work is so different from many other criminologists (both quantitative and qualitative), let alone researchers in other scientific fields (like theoretical physics or history), that I think saying we “share a common research method” is a bit of a stretch.

When my son was younger and had science fairs, they were broken into two different types of submissions; traditional science experiments, like measure a plants growth in sunlight vs without, or engineering “build things”. The academic work I am most proud of is in the engineering “build things” camp. These modest contributions in various algorithms – a few have been implemented in major software, and I know some crime analysis units using that work as well – really have nothing to do with the scientific method. Me deriving standard errors for control charts for crime trends is only finding truth in a very tautological way – I think they are useful though.

There is no bright line between my work and “activism” – I don’t think that is a bad thing though and it was the point of the work. You could probably say Janet Lauritsen is an activist for more useful national level CJ statistics. Jukka appears to me to be making normative opinions about he thinks Janet’s activism is more rigorously motivated than Vitale’s – which I agree with, but doesn’t say much if anything about the field of criminology as a whole or recent changes in the field. (If anything it is evidence against Jukka’s opinion, I posit Janet is clearly more influential in the field than Vitale.)

To end with the note “on the brink” – it may be unfair to Jukka (sometimes you don’t get to pick your titles in magazine articles). Part of the way I view being an academic and critiquing work I imagine people find irksome – it involves taking real words people say, trying to reasonably map them to statements that can be confirmed or refuted (often people say things that are quite fuzzy), and then articulating why those statements are maybe right/maybe wrong. It can seem pedantic, but I am a Popper kind-of-guy, and being able to confirm or refute statements I think is the only way we can get closer to objective truth.

To do this with “on the brink” takes more leaps than statements such as “increasingly political and intolerant”. “Criminology” is the general study of criminal behavior – which I am pretty confident will continue on as long as people commit crimes with or without the ASC yearly conference. We can probably limit the “on the brink” statement to something more specific like the American Society of Criminology on the brink. I don’t know about the ASC financials, but I am going to guess Jukka meant by this statement more of a proclamation about the legitimacy of the organization to outside groups.

I am not so sure this is the point of ASC though – it derives its value by being a social club for people who do criminology research. At least that is my impression of going to ASC conferences from my decade as a criminologist. Part of Jukka’s point is that things are getting worse more recently – you can’t lose something you never had to begin with though.

4 Comments
by Andy Wheeler on May 14, 2023  •  Permalink
Posted in Criminal Justice, scholarly
Tagged ,

Posted by Andy Wheeler on May 14, 2023

https://andrewpwheeler.com/2023/05/14/criminology-not-on-the-brink/

This one simple change will dramatically improve reproducibility in journals

So Eric Stewart is back in the news, and it appears a new investigation has prompted him to resign from Florida State. For a background on the story, I suggest reading Justin Pickett’s EconWatch article. In short, Justin did analysis of his own papers he co-authored with Stewart to show what is likely data fabrication. Various involved parties had superficial responses at first, but after some prodding many of Stewart’s papers were subsequently retracted.

So there is quite a bit of human messiness in the responses to accusations of error/fraud, but I just want to focus on one thing. In many of these instances, the flow goes something like:

  1. individual points out clear numeric flaws in a paper
  2. original author says “I need time to investigate”
  3. multiple months later, original author has still not responded
  4. parties move on (no resolution) OR conflict (people push for retraction)

My solution here is a step that mostly fixes the time lag in steps 2/3. Authors who submit quantitative results should be required to submit statistical software log files along with their article to the journal from the start.

So there is a push in social sciences to submit fully reproducible results, where an outside party can replicate 100% of the analysis. This is difficult – I work full time as a software engineer – it requires coding skills most scientists don’t have, as well as outside firms to devote resources to the validation. (Offhand, if you hired me to do this, I would probably charge something like $5k to $10k I am guessing given the scope of most journal articles in social sciences.)

An additional problem with this in criminology research, we are often working with sensitive data that cannot easily be shared.

I agree a fully 100% reproducible would be great – lets not make the perfect the enemy of the good though. What I am suggesting is that authors should directly submit the log files that they used to produce tables/regression results.

Many authors currently are running code interactively in Stata/R/SPSS/whatever, and copy-pasting the results into tables. So in response to 1) above (the finding of a data error), many parties assume it is a data transcription error, and allow the original authors leeway to go and “investigate”. If journals have the log files, it is trivial to see if a data error is a transcription error, and then can move into a more thorough forensic investigation stage if the logs don’t immediately resolve any discrepancies.

If you are asking “Andy, I don’t know how to save a log file from my statistical analysis”, here is how below. It is a very simple thing – a single action or line of code.

This is under the assumption people are doing interactive style analysis. (It is trivial to save a log file if you have created a script that is 100% reproducible, e.g. in R it would then just be something like Rscript Analysis.R > logfile.txt.) So is my advice to save a log file when doing interactive partly code/partly GUI type work.

In Stata, at the beginning of your session use the command:

log using "logfile.txt", text replace

In R, at the beginning of your session:

sink("logfile.txt")
...your code here...
# then before you exit the R session
sink()

In SPSS, at the end of your session:

OUTPUT EXPORT /PDF DOCUMENTFILE="local_path\logfile.pdf".

Or you can go to the output file and use the GUI to export the results.

In python, if you are doing an interactive REPL session, can do something like:

python > logfile.txt
...inside REPL here...

Or if you are using Jupyter notebooks can just save the notebook a html file.

If interested in learning how to code in more detail for regression analysis, I have PhD course notes on R/SPSS/Stata.

This solution is additional work from the authors perspective, but a very tiny amount. I am not asking for 100% reproducible code front to back, I just want a log file that shows the tables. These log files will not show sensitive data (just summaries), so can be shared.

This solution is not perfect. These log files can be edited. Requiring these files will also not prevent someone from doctoring data outside of the program and then running real analysis on faked data.

It ups the level of effort for faking results though by a large amount compared to the current status quo. Currently it just requires authors to doctor results in one location, this at a minimum requires two locations (and to keep the two sources equivalent is additional work). Often the outputs themselves have additional statistical summaries though, so it will be clearer if someone doctored the results than it would be from a simpler table in a peer reviewed article.

This does not 100% solve the reproducibility crisis in social sciences. It does however solve the problem of “I identified errors in your work” and “Well I need 15 months to go and check my work”. Initial checks for transcription vs more serious errors with the log files can be done by the journal or any reasonable outsider in at most a few hours of work.

Leave a comment
by Andy Wheeler on April 14, 2023  •  Permalink
Posted in Criminal Justice, Papers, Python, R, scholarly, SPSS, Stata, writing
Tagged

Posted by Andy Wheeler on April 14, 2023

https://andrewpwheeler.com/2023/04/14/this-one-simple-change-will-dramatically-improve-reproducibility-in-journals/

ASEBP blog posts, and auto screenshotting websites

I wanted to give an update here on the Criminal Justician series of blogs I have posted on the American Society of Evidence Based Policing (ASEBP) website. These include:

  • Denver’s STAR Program and Disorder Crime Reductions
    • Assessing whether Denver’s STAR alternative mental health responders can be expected to decrease a large number of low-level disorder crimes.
  • Violent crime interventions that are worth it
    • Two well-vetted methods – hot spots policing and focused deterrence – are worth the cost for police to implement to reduce violent crime.
  • Evidence Based Oversight on Police Use of Force
    • Collecting data in conjunction with clear administrative policies has strong evidence it overall reduces officer use of force.
  • We don’t know what causes widespread crime trends
    • While we can identify whether crime is rising or falling, retrospectively identifying what caused those ups and downs is much more difficult.
  • I think scoop and run is a good idea
    • Keeping your options open is typically better than restricting them. Police should have the option to take gun shot wound victims directly to the emergency room when appropriate.
  • One (well done) intervention is likely better than many
    • Piling on multiple interventions at once makes it impossible to tell if a single component is working, and is likely to have diminishing returns.

Going forward I will do a snippet on here, and refer folks to the ASEBP website. You need to sign up to be able to read that content – but it is an organization that is worth joining (besides for just reading my takes on science around policing topics).

So my CRIME De-Coder LLC has a focus on the merger of data science and policing. But I have a bit of wider potential application. Besides statistical analysis in different subject areas, one application I think will be of wider interest to public and private sector agencies is my experience in process automation. These often look like boring things – automating generating a report, sending an email, updating a dashboard, etc. But they can take substantial human labor, and automating also has the added benefit of making a process more robust.

As an example, I needed to submit my website as a PDF file to obtain a copyright. To do this, you need to take screenshots of your website and all its subsequent pages. Googling on this for selenium and python, the majority of the current solutions are out of date (due to changes in the Chrome driver in selenium over time). So here is the solution I scripted up the morning I wanted to submit the copyright – it took about 2 hours total in debugging. Note that this produces real screenshots of the website, not the print to pdf (which looks different).

It is short enough for me to just post the entire script here in a blog post:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
import time
from PIL import Image
import os

home = 'https://crimede-coder.com/'

url_list = [home,
            home + 'about',
            home + 'blog',
            home + 'contact',
            home + 'services/ProgramAnalysis',
            home + 'services/PredictiveAnalytics',
            home + 'services/ProcessAutomation',
            home + 'services/WorkloadAnalysis',
            home + 'services/CrimeAnalysisTraining',
            home + 'services/CivilLitigation',
            home + 'blogposts/2023/ServicesComparisons']

res_png = []

def save_screenshot(driver, url, path, width):
    driver.get(url)
    # Ref: https://stackoverflow.com/a/52572919/
    original_size = driver.get_window_size()
    #required_width = driver.execute_script('return document.body.parentNode.scrollWidth')
    required_width = width
    required_height = driver.execute_script('return document.body.parentNode.scrollHeight')
    driver.set_window_size(required_width,required_height)
    #driver.save_screenshot(path)  # has scrollbar
    driver.find_element(By.TAG_NAME, 'body').screenshot(path)  # avoids scrollbar
    driver.set_window_size(original_size['width'], original_size['height'])

options = Options()
options.headless = True
driver = webdriver.Chrome(options=options)

for url in url_list:
    driver.get(url)
    if url == home:
        name = "index.png"
    else:
        res_url = url.replace(home,"").replace("/","_")
        name = res_url + ".png"
    time.sleep(1)
    res_png.append(name)
    save_screenshot(driver,url,name,width=1400)

driver.quit()

# Now appending to PDF file
images = [Image.open(f).convert('RGB') for f in res_png if f[-3:] == 'png']
i1 = images.pop(0)
i1.save(r'Website.pdf', save_all=True, append_images=images)

# Now removing old PNG files
for f in res_png:
    os.remove(f)

One of the reasons I want to expand knowledge of coding practices into policing (as well as other public sector fields) is that this simple of a thing doesn’t make sense for me to package up and try to monetize. The IP involved in a 2 hour script is not worth that much. I realize most police departments won’t be able to take the code above and actually use it – it is better for your agency to simply do a small contract with me to help you automate the boring stuff.

I believe this is in large part a better path forward for many public sector agencies, as opposed to buying very expensive Software-as-a-Service solutions. It is better to have a consultant to provide a custom solution for your specific agency, than to spend money on some big tool and hope your specific problems fit their mold.

Leave a comment
by Andy Wheeler on March 14, 2023  •  Permalink
Posted in Crime Analysis, Criminal Justice, data science, Python, scholarly
Tagged ,

Posted by Andy Wheeler on March 14, 2023

https://andrewpwheeler.com/2023/03/14/asebp-blog-posts-and-auto-screenshotting-websites/

Getting access to paywalled newspaper and journal articles

So recently several individuals have asked about obtaining articles they do not have access to that I cite in my blog posts. (Here or on the American Society of Evidence Based Policing.) This is perfectly fine, but I want to share a few tricks I have learned on accessing paywalled newspaper articles and journal articles over the years.

I currently only pay for a physical Sunday newspaper for the Raleigh News & Observer (and get the online content for free because of that). Besides that I have never paid for a newspaper article or a journal article.

Newspaper paywalls

Two techniques for dealing with newspaper paywalls. 1) Some newspapers you get a free number of articles per month. To skirt this, you can open up the article in a private/incognito window on your preferred browser (or open up the article in another browser entirely, e.g. you use Chrome most of the time, but have Firefox just for this on occasion.)

If that does not work, and you have the exact address, you can check the WayBack machine. For example, here is a search for a WaPo article I linked to in last post. This works for very recent articles, so if you can stand being a few days behind, it is often listed on the WayBack machine.

Journal paywalls

Single piece of advice here, use Google Scholar. Here for example is searching for the first Braga POP Criminology article in the last post. Google scholar will tell you if a free pre or post-print URL exists somewhere. See the PDF link on the right here. (You can click around to “All 8 Versions” below the article as well, and that will sometimes lead to other open links as well.)

Quite a few papers have PDFs available, and don’t worry if it is a pre-print, they rarely substance when going into print.1

For my personal papers, I have a google spreadsheet that lists all of the pre-print URLs (as well as the replication materials for those publications).

If those do not work, you can see if your local library has access to the journal, but that is not as likely. And I still have a Uni affiliation that I can use for this (the library and getting some software cheap are the main benefits!). But if you are at that point and need access to a paper I cite, feel free to email and ask for a copy (it is not that much work).

Most academics are happy to know you want to read their work, and so it is nice to be asked to forward a copy of their paper. So feel free to email other academics as well to ask for copies (and slip in a note for them to post their post-prints to let more people have access).

The Criminal Justician and ASEBP

If you like my blog topics, please consider joining the American Society of Evidence Based Policing. To be clear I do not get paid for referrals, I just think it is a worthwhile organization doing good work. I have started a blog series (that you need a membership for to read), and post once a month. The current articles I have written are:

So if you want to read more of my work on criminal justice topics, please join the ASEBP. And it is of course a good networking resource and training center you should be interested in as well.

  1. You can also sign up for email alerts on Google Scholar for papers if you find yourself reading a particular author quite often.↩︎
Leave a comment
by Andy Wheeler on February 14, 2023  •  Permalink
Posted in ask me anything, Criminal Justice, Personal Productivity, scholarly, writing
Tagged

Posted by Andy Wheeler on February 14, 2023

https://andrewpwheeler.com/2023/02/14/getting-access-to-paywalled-newspaper-and-journal-articles/

Where are they now? Job outcomes for recent SUNY crim Phds

The other day I noticed one of my PhD cohort mates, like me, took a private sector data science job. So of the 6 that finished their Phds in my cohort, 2 of us are now in private sector and the rest are professors. I was curious the overall rate for a larger sample.

There is probably some better official source, but I was able to do a search in Proquest dissertations (SUNY we needed to submit it there), for "State University of New York at Albany" AND "School of Criminal Justice" published between 2010 through 2020 and it scooped up a pretty good sample (with a few false positives I eliminated). I then added in a few people I noticed missing in that set, in the end 69 total over the 11 years (6 defenses per year actually seemed high to me). (Using the WayBack machine you can look at old Phd profiles or the old list of dissertations, but I am not sure of the completeness of either.) Then I filled in their current main job best I could into professor, private sector, university research center, think tank, government (and a few I did not even hazard a guess), based on LinkedIn/google searches/personal knowledge.

Here is the spreadsheet, let me know if you think I miscategorized you or your dissertation is missing altogether. Filtering based on the year of the dissertation is not the same as cohort (you could have started along time ago and more recently defended), but looks to me a pretty reasonable sample of “recent” Phd’s from SUNY Albany Criminal Justice program. Also missing at this Proquest search phase is likely to be missing at random (the few who were not scooped up in my search I see no reason to think are systematic based on Proquest’s idiosyncratic search). But missing in terms of me being able to look once you are in the sample is not (since if you are a professor you probably come up in a general google search for your university).

I tended to be liberal for who I listed as professor (this includes temp teaching jobs and postdocs, but not people who are adjuncts). Many people not in the professor list though were formerly professors (myself included), but tried to figure out the current main job for individuals.

The breakdown for the 69 dissertations is then:

Prof          34  49%
Gov           18  26%
Private        6   9%
Univ Research  3   4%
Think Tank     1   1%
Don't Know     7  10%

So private sector is lower overall than in my cohort, only 10% over the time period (and highest possible sample estimate is 19%, if all 7 don’t know are actually in private sector). Government jobs being at 26% I don’t find surprising, think tank and private is lower than I would have guessed though.

But from this I take away around 50% of recent PhDs in criminal justice from SUNY go on to be professors. For prospective PhDs, this estimate is also conditional on completing the PhD (they aren’t in the sample if they did not finish). If you include those individuals Gov/Private would go up in overall proportions.

Again if missing in the list or miscategorized let me know and I will update the post.

4 Comments
by Andy Wheeler on January 8, 2023  •  Permalink
Posted in Criminal Justice, scholarly
Tagged

Posted by Andy Wheeler on January 8, 2023

https://andrewpwheeler.com/2023/01/08/where-are-they-now-job-outcomes-for-recent-suny-crim-phds/

Preprint: Analysis of LED street light conversions on firearm crimes in Dallas, Texas

I have a new pre-print out, Analysis of LED street light conversions on firearm crimes in Dallas, Texas. This work was conducted in collaboration with the Child Poverty Action Lab, in reference to the Dallas Taskforce report. Instead of installing the new lights though at hotspots that CPAL suggested, Dallas stepped up conversion of street lamps to LED. Here is the temporal number of conversions over time:

And here is an aggregated quadrat map at quarter square mile grid cells (of the total number of LED conversions):

I use a diff-in-diff design (compare firearm crimes in daytime vs nighttime) to test whether the cumulative LED conversions led to reduced firearm crimes at nighttime. Overall I don’t find any compelling evidence that firearm crimes were reduced post LED installs (for a single effect or looking at spatial heterogeneity). This graph shows in the aggregate the DiD parallel trends assumption holds citywide (on the log scale), but the identification strategy really relies on the DiD assumption within each grid cell (any good advice for graphically showing that with noisy low count data for many units I am all ears!).

For now just wanted to share the pre-print. To publish in peer-review I would need to do a bunch more work to get the lit review where most CJ reviewers would want it. Also want to work on spatial covariance adjustments (similar to here, but for GLM models). Have some R code started for that, but needs much more work/testing before ready for primetime. (Although as I say in the pre-print, these should just make standard errors larger, they won’t impact the point estimates.)

So no guarantees that will be done in anytime in the near future. But no reason to not share the pre-print in the meantime.

Leave a comment
by Andy Wheeler on November 17, 2022  •  Permalink
Posted in Crime Analysis, Crime Mapping, Criminal Justice, Mapping, scholarly
Tagged , ,

Posted by Andy Wheeler on November 17, 2022

https://andrewpwheeler.com/2022/11/17/preprint-analysis-of-led-street-light-conversions-on-firearm-crimes-in-dallas-texas/

NIJ grants funding gun violence research

Before I get into the nitty gritty of this post, a few notes. First, my next post in the Criminal Justician series on ASEBP is up, Violent Crime Interventions That are Worth it. I discuss more of the costs with implementing hot spots policing and focussed deterrence from the police departments perspective, and why they are clearly worthwhile investments for many police departments facing violence problems.

Second, I want to point folks to Jacob Kaplan’s blog, most recent post The Covid Kings of Salami. Some of Jacob’s thoughts I disagree with (I think smaller papers are OK, or that policing what is big enough is a waste of time). But if you like my posts on CJ topics, you should check out Jacob’s as well.

Now onto the title – a work in progress at the moment, but working with Scott Jacques on the openness of funded US criminology research. A short post in response to the oft mistaken idea that gun violence research is banned in the US. This is confused logic related to the Dickey act saying awards for gun control advocacy are banned as being federally funded by the CDC.

There are other agencies who fund gun violence research, in particular here I have scraped data from the National Institute of Justice (what I think is likely to be the largest funder in this area). Here is some python code showing some analyses of those awards.

So first, here you can download and see the size of the scraped dataset of NIJ awards:

import pandas as pd

# award data scraped, stay tuned for code for that!
award_url = 'https://dl.dropbox.com/s/eon4iokv0qpllgl/NIJ_Awards.csv?dl=0'
award_df = pd.read_csv(award_url)
print(award_df.shape)
print(award_df['award_text'][0])

So as a first blush check for awards related to gun violence, we can just search the text for the award narrative for relevant terms, here I just search for GUN VIOLENCE and FIREARM. A more thorough investigation would either code the 7k awards or the original solicitations, but I think this will likely be largely accurate (probably slightly more false positives than false negatives).

award_df['award_textU'] = award_df['award_text'].str.upper()

# Lets try to find any of these (other text?)
word_list = ['GUN VIOLENCE','FIREARM']

for w in word_list:
    award_df[w] = 1*(award_df['award_textU'].str.find(w) > -1)

award_df['AnyGun'] = 1*(award_df[word_list].sum(axis=1) > 0)
print(award_df['AnyGun'].sum())

So we can see that we have 1,082 awards related to gun violence (out of 7,215 listed by the NIJ). Lets check out the total funding for these awards:

# Lets figure out the total allocated
award_df['AwardVal'] = award_df['field-award-amount'].str.strip()
award_df['AwardVal'] = award_df['AwardVal'].replace('[\$,]', '', regex=True)
award_df['AwardVal'] = pd.to_numeric(award_df['AwardVal'])
award_df['Tot'] = 1

cf = ['Tot','AwardVal']
award_df.groupby('AnyGun',as_index=False)[cf].sum()

So we have in the listed awards (that go back to 1998 but appear more consistently filled in starting in 2002), over 300 million in grant awards related to gun violence/firearm research. Here we can see the breakdown over time.

# See awards over time
gun_awards = award_df[award_df['AnyGun'] == 1].copy()
gun_awards.groupby('field-fiscal-year',as_index=False)[cf].sum()

So the awards gifted by NIJ no doubt have a different flavor/orientation than if you had the same money from CDC. (There are other orgs though, like NSF, who I am sure have funded research projects relevant to gun violence over time as well.) Sometimes people distinguish between “public health” vs “criminal justice” approaches, but this is a pretty superficial dichotomy (plenty of people in public health have gotten NIJ awards).

So you certainly could argue the Dickey amendment changed the nature of gun violence research being conducted. And since the CDC budget is so massive, I suppose you could argue that it reduced the overall amounts of gun violence research being funded (although it is likely 0 sum, more for firearm research would have slashed some other area). You could use the same argument to say NIJ though is underfunded instead of advocating for the CDC to write the checks though.

But the stronger statement I often see stated, that firearm research is entirely banned in the US, is not even close to being correct.

Leave a comment
by Andy Wheeler on November 15, 2022  •  Permalink
Posted in Criminal Justice, Python, scholarly
Tagged , ,

Posted by Andy Wheeler on November 15, 2022

https://andrewpwheeler.com/2022/11/15/nij-grants-funding-gun-violence-research/

Outputs vs Outcomes and Agile

For my criminal justice followers, there is a project planning strategy, Agile, that dominates software engineering. The idea behind Agile is to formulate plans in short sprints (we do two week sprints at my work). So we have very broad based objectives (Epics) that can span a significant amount of time. Then we have shorter goals (Stories) that are intended to take up the sprint. Within each story, we further break down our work into specific tasks that we can estimate how long they will take. So something at my work may look like:

  • Build Model to Predict Readmission for Heart Attacks (Epic)
    • Create date pipeline for training data (Story)
      • SQL functions to prepare data (Task, 2 days)
      • python code to paramaterize SQL (Task, 3 days)
      • Unit tests for python code (Task, 1 day)
    • Build ML Model (Story)
      • evaluate different prediction models (Task, 2 days)
      • …
    • Deploy ML Model in production (Story)
      • …

Etc. People at this point often compare Agile vs Waterfall, where waterfall is more longish term planning (often on say a quarterly schedule). And Agile per its name is suppossed to be more flexible, and modify plans on short term. Most of my problems with Agile could apply though to Waterfall planning as well – short term project planning (almost by its nature) has to be almost solely focused on outputs and not outcomes.

Folks with a CJ background will know what I am talking about here. So police management systems often contrast focusing on easily quantifiable outputs, such as racking up traffic tickets and low level arrests, vs achieving real outcomes, such as increased traffic safety or reducing violent crime. While telling police officers to never do these things does not make sense, you can give feedback/nudge them to engage in higher quality short term outputs that should better promote those longer term outcomes you want.

Agile boards (where we post these Epics/Stories/Tasks, for people to keep tabs on what everyone is doing) are just littered with outputs that have little to no tangible connection to real life outcomes. Take my Heart Attack example. It may be there is a current Heart Attack prediction system in place based on a simple scorecard – utility in that case would be me comparing how much better my system is than the simpler scorecard method. If we are evaluating via dollars and cents, it may only make sense to evaluate how effective my system is in promoting better health outcomes (e.g. evaluating how well my predictive system reduces follow up heart attacks or some other measure of health outcomes).

The former example is not a unit of time (and so counts for nothing in the Agile framework). Although in reality it should be the first thing you do (and drop the project if you cannot sufficiently beat a simple baseline). You don’t get brownie points for failing fast in this framework though. In fact you look bad, as you did not deliver on a particular product.

The latter example unfortunately cannot be done in a short time period – we are often talking about timescales of years at that point instead of weeks. People can look uber productive on their Agile board, and can easily accomplish nothing of value over broad periods of time.

Writing this post as we are going through our yearly crisis of “we don’t do Agile right” at my workplace. There are other more daily struggles with Agile – who defines what counts as meeting an objective? Are we being sufficiently specific in our task documentation? Are people over/under worked on different parts of the team? Are we estimating the time it takes to do certain tasks accurately? Does our estimate include actual work, or folds in uncertainty due to things other teams are responsible for?

These short term crises of “we aren’t doing Agile right” totally miss the boat for me though. I formulate my work strategy by defining end goals, and then work backwards to plan the incremental outputs necessary to achieve those end goals. The incremental outputs are a means to that end goal, not the ends themselves. I don’t really care if you don’t fill out your short term tasks or mis-estimate something to take a week instead of a day – I (and the business) cares about the value added of the software/models you are building. It isn’t clear to me that looking good on your Agile board helps accomplish that.

1 Comment
by Andy Wheeler on September 30, 2022  •  Permalink
Posted in Criminal Justice, data science, Personal Productivity, scholarly
Tagged ,

Posted by Andy Wheeler on September 30, 2022

https://andrewpwheeler.com/2022/09/30/outputs-vs-outcomes-and-agile/

Gun Buy Back Programs Probably Don’t Work

When I was still a criminology professor, I remember one day while out getting groceries receiving a cold call from a police department interested in collaborating. They asked if I could provide evidence to support their cities plan to implement sex offender residence restrictions. While taking the call I was walking past a stand for the DARE program.

A bit of inside pool for my criminology friends, but for others these are programs that have clearly been shown to not be effective. Sex offender restrictions have no evidence they reduce crimes, and DARE has very good evidence it does not work (and some mild evidence it causes iatrogenic effects – i.e. causes increased drug use among teenagers exposed to the program).

This isn’t a critique of the PD who called me – academics just don’t do a great job of getting the word out. (And maybe we can’t effectively, maybe PDs need to have inhouse people do something like the American Society of Evidence Based Policing course.)

One of the programs that is similar in terms of being popular (but sparse on evidence supporting it) are gun buy back programs. Despite little evidence that they are effective, cities still continue to support these programs. Both Durham and Raleigh recently implemented buy backs for example.

What is a gun buy back program? Police departments encourage people to turn in guns – no questions asked – and they get back money/giftcards for the firearms (often in the range of $50 to $200). The logic behind such programs is that by turning in firearms it prevents them from being used in subsequent crimes (or suicides). No questions asked is to encourage individuals who have even used the guns in a criminal manner to not be deterred from turning in the weapons.

There are not any meta-analyses of these programs, but the closest thing to it, a multi-city study by Ferrazares et al. (2021), analyzing over 300 gun buy backs does not find macro, city level evidence of reduced gun crimes subsequent to buy back programs. While one can cherry pick individual studies that have some evidence of efficacy (Braga & Wintemute, 2013; Phillips et al., 2013), the way these programs are typically run in the US they are probably not effective at reducing gun crime.

Lets go back to first principles – if we 100% knew a gun would be used in the commission of a crime, then “buying” that gun would likely be worth it. (You could say an inelastic criminal will find or maybe even purchase a new gun with the reward, Mullin (2001), so that purchase does not prevent any future crimes, but I am ignoring that here.)

We do not know that for sure any gun will be used in the commission of a crime – but lets try to put some guesstimates on the probability that it will be used in a crime. There are actually more guns in the US than there are people. But lets go with a low end total of 300 million guns (Braga & Wintemute, 2013). There are around half a million crimes committed with a firearm each year (Planty et al., 2013). So that gives us 500,000/300,000,000 ~ 1/600. So I would guess if you randomly confiscated 600 guns in the US, you would prevent 1 firearm crime.

This has things that may underestimate (one gun can be involved in multiple crimes, still the expected number of crimes prevented is the same), and others that overestimate (more guns, fewer violent crimes, and replacement as mentioned earlier). But I think that this estimate is ballpark reasonable – so lets say 500-1000 guns to reduce 1 firearm crime. If we are giving out $200 gift cards per weapon returned, that means we need to drop $100k to $200k to prevent one firearm crime.

Note I am saying one firearm crime (not homicide), if we were talking about preventing one homicide with $200k, that is probably worth it. That is not a real great return on investment though for the more general firearm crimes, which have costs to society typically in the lower 5 digit range.

Gun buy backs have a few things going against them though even in this calculation. First, the guns returned are not a random sample of guns. They tend to be older, long guns, and often not working (Kuhn et al., 2021). It is very likely the probability those specific guns would be used in the commission of a crime is smaller than 1/600. Second is just the pure scope of the programs, they are often just around a few hundred firearms turned in for any particular city. This is just too small a number to reasonably tell whether they are effective (and what makes the Australian case so different).

Gun buy backs are popular, and plausibly may be “worth it”. (If encouraging working hand guns (Braga & Wintemute, 2013) and the dollar rewards are more like $25-$50 the program is more palatable in my mind in terms of at least potentially being worth it from a cost/benefit perspective.) But with the way most of these studies are conducted, they are hopeless to identify any meaningful macro level crime reductions (at the city level, would need to be more like 20 times larger in scope to notice reductions relative to typical background variation). So I think more proven strategies, such as focussed deterrence or focusing on chronic offenders, are likely better investments for cities/police departments to make instead of gun buy backs.

References

3 Comments
by Andy Wheeler on August 21, 2022  •  Permalink
Posted in Crime Analysis, Criminal Justice, scholarly
Tagged ,

Posted by Andy Wheeler on August 21, 2022

https://andrewpwheeler.com/2022/08/21/gun-buy-back-programs-probably-dont-work/

« Older Posts
Newer Posts »