by Lexi Boyer
April 2015
Real-time job data, job ads compiled from numerous online job boards, are a relatively new source of labor market information that may help improve labor market performance and efficiency by providing more timely measures of the demand for specific occupations, skill sets, certifications, and licenses. For this reason real-time data has become a hot topic among states, government agencies, schools, and businesses hoping to use these data in order to make better-informed policy decisions. However, very little analysis has been done about how well real-time data represent the real population of job vacancies over time and how they compare to traditional labor market information.
This analysis uses data from the fourth quarter of 2005, second quarter of 2014, and every second and fourth quarter in between. The data come from a comparison of the Conference Board Help Wanted Online Data Series (HWOL) and the Minnesota Job Vacancy Survey (JVS) to determine whether HWOL would be an appropriate replacement for the entire JVS, only sections of the JVS, or as a supplement. This article also surveys the strengths and weaknesses of HWOL.
Questionnaires were sent to a stratified sample of 10,000 firms in the 13 Economic Development regions of Minnesota. Firms that are private households, personnel service industry establishments, or businesses with no employees at the time the sample was pulled were excluded from the sampling. For the purposes of this survey, a job vacancy is defined as a vacancy that is open-for-hire at the time the survey was conducted. Data were collected by mail, phone, and email. When appropriate, data were also collected from the firm's website if job vacancy information was present. Information collected included job title, whether the opening was for a full-time or part-time position, wages, number of vacancies, whether a license was required, and the education and amount of experience required.
A special request was made to obtain monthly HWOL data containing information on the volume of job postings for every second and fourth quarter between the fourth quarter of 2005 and the second quarter of 2014. The following variables were included in the data received:
HWOL does not provide information on education or wages since only about 30 percent of the job ads indicate a desired education level and only 15 percent of job ads indicate a wage. These data included anonymous and staffing firm ads as well as Craigslist ads. Craigslist ads, however, were carefully cleaned to ensure improper ads were removed.5 HWOL obtains its data from WANTED Analytics which is their data provider.6 Monthly HWOL data were collected using a mid-month survey reference period which means the volume of online job ads for any particular month is the sum of all the job ads posted between the 14th of the prior month and the 13th of the month of interest. In order to compare the state level data obtained from HWOL to the JVS data, the HWOL data had to be sorted by region as well as by 2-, 3- and 6-digit SOC and then collapsed to the quarter level. Thus, the data were summed by SOC for each quarter separately for both the state and planning region level. The average of the volume over each three-month period for each quarter was then divided by three to smooth the fluctuations caused by strong seasonality in the HWOL data.
It is important to note that the time period of the HWOL data does not perfectly match up with the time period of the JVS data as the HWOL data were collected using a mid-month reference period while the JVS data refer to the time the questionnaire was filled out. In addition, the HWOL job ad "volume new" average over a three- month period is being compared to the quarterly JVS job vacancy volume. For the purposes of this study, JVS job vacancies are treated as true vacancies while HWOL job ads are treated as possible openings since some employers post job ads in order to have a constant applicant pool or to gauge market competitiveness. Therefore, a job ad does not equal a job opening.
Table 1 describes the 23 major occupational groups and Table 2 provides HWOL and JVS comparison information for the Twin Cities7 and Greater Minnesota areas. Using the JVS data as the baseline, HWOL appears to favor high-skill jobs such as those in management (11-0000), computer (15-0000), and legal occupations (23-0000). Like many high-skill occupations, job openings for these occupations are more likely to be posted online. One reason for this may be that firms hiring in these areas have access to a larger pool of potential job candidates by posting a job vacancy online, which allows firms to recruit more widely and be more selective about the individuals they hire. Another reason may be that high-skill workers are more likely to have access to the Internet and use it for job search.
| Table 1: 2-Digit SOC | |
|---|---|
| SOC | Description |
| 11-0000 | Management Occupations |
| 13-0000 | Business and Financial Operations Occupations |
| 15-0000 | Computer and Mathematical Occupations |
| 17-0000 | Architecture and Engineering Occupations |
| 19-0000 | Life, Physical, and Social Science Occupations |
| 21-0000 | Community and Social Service Occupations |
| 23-0000 | Legal Occupations |
| 25-0000 | Education, Training, and Library Occupations |
| 27-0000 | Arts, Design, Entertainment, Sports, and Media Occupations |
| 29-0000 | Healthcare Practitioners and Technical Occupations |
| 31-0000 | Healthcare Support Occupations |
| 33-0000 | Protective Service Occupations |
| 35-0000 | Food Preparation and Serving Related Occupations |
| 37-0000 | Building and Grounds Cleaning and Maintenance Occupations |
| 39-0000 | Personal Care and Service Occupations |
| 41-0000 | Sales and Related Occupations |
| 43-0000 | Office and Administrative Support Occupations |
| 45-0000 | Farming, Fishing, and Forestry Occupations |
| 47-0000 | Construction and Extraction Occupations |
| 49-0000 | Installation, Maintenance, and Repair Occupations |
| 51-0000 | Production Occupations |
| 53-0000 | Transportation and Material Moving Occupations |
| 55-0000 | Military Specific Occupations |
| Source: Bureau of Labor Statistics 2010 | |
On the other hand, JVS appears to favor low-skill jobs. For example, the JVS volumes for low-skill occupations such as sales and related occupations (41-0000), health support occupations (31-0000), and food preparation and serving related occupations (35-0000) are significantly higher than the HWOL volumes. One reason for this is that firms tend to post openings for low-skill occupations on the company's bulletin board, window, or door, in the local newspapers, or on the firm's website. Interestingly, education, training, and library occupations also tended to be better captured by the JVS than by HWOL.
Further research shows one reason HWOL may under-represent the number of openings for education, training, and library occupations is that these openings tend to be posted on school or library websites rather than on job boards. Similarly, healthcare support occupation openings tend to be posted on nursing home websites rather than on job boards. In the case of healthcare support occupations such as home health aides, many agencies do not post openings but rather request candidates to send in their resumes. Individuals are then contacted when positions become available. Because HWOL collects only online postings, it under-represents occupations and industries that use recruitment methods other than online job posts.
The prevalence of online job postings differs significantly between the metro and the Greater Minnesota area. In particular, the ratio of online job postings to job vacancies is roughly 99.5 percent in the Twin Cities area but only 47.4 percent in Greater Minnesota so that job openings in the Twin Cities are roughly twice as likely to be posted online than job openings in Greater Minnesota. To identify the reasons behind this difference in propensity to post online, data were compiled by major occupational groups at the 2-digit SOC level for both the metro and the Greater Minnesota areas (Table 2). During the JVS rounds between 2005 fourth quarter and 2014 second quarter, 6.5 percent of all vacancies reported in the Twin Cities were in IT occupations (15-0000), 5.7 percent in management occupations (11-0000), 7.1 percent in business and financial operations occupations (13-0000), and 0.4 percent in legal occupations (23-0000) compared to 1.4 percent in IT, 2.3 percent in management, 2.2 percent in business and financial operations, and 0.1 percent in legal occupations of all the vacancies reported in Greater Minnesota. On the other hand, there are greater shares of vacancies in Greater Minnesota in occupations that are less likely to be posted online. Examples of these occupations include food preparation and serving related occupations (35-0000) with 8.3 vs. 13.0 percent, healthcare support occupations (31-0000) with 5.3 vs. 8.3 percent, and personal care and service occupations (39-0000) with 4.2 vs. 4.5 percent. As seen from the last column in Table 2, the Metro/Greater MN ratio is greater than 1 across all occupation groups; this indicates that a job from any given occupation group is more likely to be posted online in the Twin Cities than in the Greater Minnesota area. For example, a job opening in management (11-000) is 1.2 times more likely to be posted online in the Twin Cities than in Greater Minnesota and a job opening in a computer and mathematical occupation is 1.5 times more likely to be posted online in the Twin Cities than in Greater Minnesota. These results suggest that the gap of online job posts between the Twin Cities and Greater Minnesota is driven more by the practices of employers than occupation distributions.
| Table 2: Greater Minnesota vs. Twin Cities | |||||||||
|---|---|---|---|---|---|---|---|---|---|
SOC |
Greater Minnesota | Twin Cities | |||||||
| HWOL/JVS | HWOL Share | JVS Share of Total | HWOL Share of Total | HWOL/JVS | HWOL Share | JVS Share of Total | HWOL Share of Total | Metro/Greater MN Ratio | |
| 11-0000 | 137.1% | 57.8% | 2.3% | 6.6% | 163.6% | 62.1% | 5.7% | 9.4% | 1.2 |
| 13-0000 | 83.9% | 45.6% | 2.2% | 3.8% | 114.4% | 53.4% | 7.1% | 8.2% | 1.4 |
| 15-0000 | 155.9% | 60.9% | 1.4% | 4.4% | 235.2% | 70.2% | 6.5% | 15.3% | 1.5 |
| 17-0000 | 87.4% | 46.6% | 1.7% | 3.1% | 143.5% | 58.9% | 2.9% | 4.2% | 1.6 |
| 19-0000 | 47.4% | 32.1% | 0.9% | 0.9% | 51.7% | 34.1% | 1.6% | 0.8% | 1.1 |
| 21-0000 | 74.5% | 42.7% | 2.0% | 3.1% | 90.6% | 47.5% | 1.9% | 1.8% | 1.2 |
| 23-0000 | 103.6% | 50.9% | 0.1% | 0.3% | 212.7% | 68.0% | 0.4% | 0.8% | 2.1 |
| 25-0000 | 44.0% | 30.6% | 3.9% | 3.6% | 55.3% | 35.6% | 4.8% | 2.6% | 1.3 |
| 27-0000 | 79.9% | 44.4% | 1.4% | 2.4% | 137.9% | 58.0% | 1.8% | 2.5% | 1.7 |
| 29-0000 | 64.9% | 39.4% | 7.7% | 10.6% | 73.7% | 42.4% | 7.4% | 5.5% | 1.1 |
| 31-0000 | 20.2% | 16.8% | 8.3% | 3.5% | 37.7% | 27.4% | 5.3% | 2.0% | 1.9 |
| 33-0000 | 34.9% | 25.9% | 1.1% | 0.8% | 48.9% | 32.8% | 1.0% | 0.5% | 1.4 |
| 35-0000 | 14.2% | 12.5% | 13.0% | 3.9% | 40.9% | 29.0% | 8.3% | 3.4% | 2.9 |
| 37-0000 | 30.7% | 23.5% | 4.6% | 2.9% | 64.4% | 39.2% | 2.9% | 1.8% | 2.1 |
| 39-0000 | 22.9% | 18.7% | 4.5% | 2.2% | 44.4% | 30.7% | 4.2% | 1.9% | 1.9 |
| 41-0000 | 46.5% | 31.7% | 12.1% | 11.9% | 88.2% | 46.9% | 13.3% | 11.8% | 1.9 |
| 43-0000 | 75.3% | 42.9% | 8.0% | 12.8% | 120.4% | 54.6% | 11.3% | 13.7% | 1.6 |
| 45-0000 | 17.2% | 14.7% | 1.4% | 0.5% | 33.9% | 25.3% | 0.2% | 0.1% | 2.0 |
| 47-0000 | 37.8% | 27.4% | 4.5% | 3.6% | 172.6% | 63.3% | 1.4% | 2.3% | 4.6 |
| 49-0000 | 75.7% | 43.1% | 3.2% | 5.2% | 140.3% | 58.4% | 2.2% | 3.1% | 1.9 |
| 51-0000 | 40.8% | 29.0% | 8.0% | 6.9% | 113.7% | 53.2% | 4.2% | 4.7% | 2.8 |
| 53-0000 | 42.3% | 29.7% | 7.8% | 7.0% | 59.4% | 37.3% | 5.7% | 3.4% | 1.4 |
| 55-0000 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A |
| Grand Total | 47.4% | 32.2% | 100.0% | 100.0% | 99.5% | 49.9% | 100.0% | 100.0% | 2.1 |
| Source: DEED analysis using data from The Conference Board's Help Wanted Online Data Series and the Minnesota Job Vacancy Survey. | |||||||||
Many states appear to use real-time labor market information as an indication of real-time labor demand. To support this usage of information, states suggest there is a relationship between the number of job postings and the number of new hires. Unfortunately, the data provided were not sufficient to analyze whether a correlation existed between the volume of job postings and the volume of new hires. Moreover, caution should be used when using real-time job ad data as a measure of growth in occupations as the findings of this study suggest that not all occupational groups are represented equally. Further caution should be used when looking at the volume of online job ads in the very short-run as the volume of postings can be misleading because employers or recruiting agencies tend to post ads only once a week or all at the beginning or end of the week. This is especially true if employers or recruiting agencies post ads only once a week or all at the beginning or end of the week. Thus, the volume of online job ads can be quite volatile with spikes in volume resulting when a recruiting agency or an employer posts an ad. Future research needs to be done to understand better the relationship between the volume of online job postings and the volume of new hires and whether there is a strong relationship between the two for certain occupations or occupational groups.
Overall, it is not recommended that HWOL be used as a replacement for the entire JVS or parts of the JVS. The findings using these new data suggest that relying solely on HWOL as an indicator of the occupational distribution of job opportunities and the skills associated with them will skew our understanding of the current needs of the Minnesota labor market. Likewise, it will skew our training programs toward occupations more likely to be posted online and will produce a labor workforce that better suits the needs of the Twin Cities metro area than the needs of the Greater Minnesota. Any use of HWOL or other real-time data should be used in conjunction with traditional LMI data. The application and interpretation of real-time LMI data will likely provide better results when used by someone who understands its limitation.
1 These are based on the Standard Occupational Classification (SOC) codes, but independently expanded for more detail.
2 Not all ads are coded to a county. According to The Conference Board Help Wanted Online Data Series Technical Notes 2012 version, roughly 93 percent of all ads are coded to a county/city level. The remaining 7 percent is made up of statewide and nationwide ads where 5 percent are coded as "Statewide" and 2 percent are coded as "nationwide" ads.
3 Volume new, a subset of volume, refers to ads that have not been seen for at least a set period of time. For the purposes of this analysis, volume new was used instead of volume.
4 Volume refers to the total volume of ads spidered (spiders are internet bots that crawl the web collecting job ad information) between the 14th of the prior month and the 13th of the month of interest.
5 Craigslist is one of the largest job boards in the U.S. It is also a large driver of anonymous ads because it does not provide a field for employer name.
6 WANTED Analytics collects its data by spidering job boards, online newspaper ads, government job boards, and corporate job boards. Spiders are internet bots that crawl the web collecting job ad information. The process takes place 24/7 for HWOL and needs roughly two days to spider a job board.
7 The Twin Cities metro area is defined as the seven-county Minneapolis-St. Paul region which includes Anoka, Carver, Dakota, Hennepin, Ramsey, Scott, and Washington Counties.
