Search results matching tag 'Strategies'

"How Technology Enables Transformation of Human Service Administration" by M. Geffen and J. Kost

jstengel — Mon, 16 Feb 2009 15:29:24 GMT

This article focuses on human service organizations whose ultimate service mission to public clientele is attainment of self-sufficiency. The authors cite various reasons for an ever-evolving 21^st Century human services model of enormous complexity that must exponentially increase its service delivery capability to keep up with increasing and varied client demands; demands which, in turn, are fraught with an increase in the pace and frequency of individual needs. Thus, that ultimate goal of users of social services realistically achieving self-sufficiency becomes still more elusive. Simply bisected, the current service delivery evolution supports innovative change on two orchestrated flanks: (1) technology and (2) human support. The latter flank must undergo reverberating changes in front-line service worker jobs and that of their case managers as a result of admittedly late-arriving technical changes in the former flank.

The authors offer a well-defined “categorical grip” to interested readers for getting a conceptual handle on the technologically-based tools involved in exponentially keeping apace of diverse human service needs: integrated case management solutions (e.g., residing on CRM platforms), sophisticated eligibility (etc) rules engines, workflow tools that integrate and make seamless multiple (triaged) services, knowledge management and collaboration tools, computer tablets and other devices for users’ remote access to services, and advanced data integration/business intelligence tools providing real-time data where imminently required.

Some of this recommended technology comes by way of budget-friendly COTS (commercial off-the-shelf) software and hardware; some of which also lends itself to increasing client participation in needed services (self-sufficiency), while raising the service performance bar for human service agencies. The authors add three recommended caveats: (1) overhauling the procurement process (which inhibits multiyear turnkey projects in general); (2) acting on the foreknowledge that vendors of COTS solutions are as important as the products they promote (i.e., issues regarding installation, integration, and maintenance); and (3) having internal technical support readily available to create easier and more meaningful ways for diverse users to use an upgraded system to its fullest (operations side of knowledge management).

Citation

Geffen, M. and J. Kost (2006). "How Technology Enables Transformation of Human Service Administration." Policy & Practice of Public Human Services 64(4): 14-17.

"The discipline of innovation" by Peter Drucker

jstengel — Thu, 12 Feb 2009 15:19:18 GMT

Most successful innovations, according to Drucker, come from a conscious, purposeful search for innovation opportunities. He saw four areas of opportunity inside organizations: unexpected occurrences, incongruities, process needs, and industry and market changes. Opportunities also exist outside organizations in its social and intellectual environments: demographic changes, changes in perception, and new knowledge. Together these seven account for almost all opportunities for innovation.

Effective innovation is simple and focused. If it tries to do too many things, people will get confused and implementation will be compromised. He also states that innovation is hard, focused, purposeful work. It requires knowledge and focus, and often, requires ingenuity. The foundation of entrepreneurship is the practice of systematic innovation.

Peter Drucker was a university professor, writer, and business guru (Wikipedia). This article was adapted from Drucker’s book Innovation and Entrepreneurship: Practice and Principles, Harper and Row, 1985.

Citation

Drucker, P. (2002). The discipline of innovation. Harvard Business Review: 95-102

"Improving the Transition from Basic Efficacy Research to Effectiveness Studies: Methodological Issues and Procedures" by Gregory N. Clarke

jstengel — Tue, 10 Feb 2009 15:16:16 GMT

This article was inspired by Weisz, Weiss, and Donenberg’s (1992) look at positive child psychotherapy effects in research suggesting “…a key task for researchers [is]….identifying those proper conditions under which effects of child therapy may be optimized.” The authors suggest controlled methodological experiment changes to enhance generalizeability in effectiveness trials in clinical/service delivery settings. It calls for including methodologies to transfer desirable aspects of efficacy research (e.g., greater independent variable control) into combined efficacy-effectiveness trials. The five broad changes to research/design methods/procedures are summarized in the last two paragraphs below.

Modified and/or extended research methods and procedures are proposed to bridge the current generalizeability gap between original “purist” laboratory efficacy research and the real-world’s “dirtier” effectiveness trials being replicated in clinics and service settings; this paper illuminates a thick line existing between the laboratory research and field-replicated research. In addition to a sample of five research methods/procedural factors that contribute to a lab-real world practice generalizeability gap/chasm (addressed by this paper), Clarke mentions other factors demanding research-design attention: therapist training; monitoring and protocol compliance; multiple therapist roles; participant recruitment methods; professional vs. nonprofessional therapists; measurement technology; participant-therapist-assessor masking to therapy condition; service setting; and participant assignment.

The five research design methods/procedures covered by this paper are summarized according to their real world application problem(a) and the author’s recommended modified research design solution(b). [1] Degree of therapy structure – (a) the problem is implementation: clinicians won’t show fidelity to regimented, manualized treatment scripts as they too often resist “rigid” or uni-modality interventions in favor of being “eclectic” with their clients on a session-by-session basis. (b) Original efficacy study researchers should provide stratified study designs of minimal protocol deviation vs. minor-moderate protocol deviation vs. non-manualized replication. [2] Integrated vs. isolated services – (a) original efficacy studies typically target one disorder, diagnosis, or problem domain; also, they typically provide one intervention under study. This “insularity” is not feasible in community treatment settings, school clinics, etc. (b) Design an integrated intervention regime of varying multiple research-based intervention “tracks” across a variety of researched disorders. [3] Usual care vs. no-treatment or placebo-attention controls – (a) the falsity of these experimental control conditions is apparent since they do not reflect what typically happens to patients who are not provided treatment. Also, efficacy studies may incur higher drop-out rates among enforced no-treatment and waiting-list patients (introducing potential bias). (b) What some studies are doing already: provide randomized “usual care” as the controls (i.e., defined as minimal treatment), which more closely (and ethically) mirrors real world conditions.

[4] Sample representativeness, sample homogeneity vs. real world population heterogeneity – (a) efficacy research subjects may be very different from their community services counterparts—making the results highly suspect. The psychological and social services world is characterized by frequent instances of co-morbidity and patients receiving services from multiple providers, etc. (b) Apply a two-tiered participant recruitment strategy in efficacy studies (Clarke’s “donut model”): (1) one homogeneous sample constitutes the donut hole; another sample (2) constitutes an unselected, heterogeneous, co-morbid donut ring—using a factorial research design. [5] Treatment parameters of dosage, modality, location, implementation – (a & b) possibly the most important issue, this one continues to seek answers to questions put by legislators and policy-makers: Does psychotherapy work? Are patients with ____ better treated in inpatient or outpatient facilities? And at what costs? For how long? In groups or individual therapy, or both? Delivered by professionals or paraprofessionals? Etc. Thus Clarke is a strong proponent of treatment process research, where sub-studies are embedded within larger efficacy studies so as to better inform and represent the users of research findings.

Citation

Clarke, G.N.(1995). Improving the Transition from Basic Efficacy Research to Effectiveness Studies: Methodological Issues and Procedures. Journal of Consulting and Clinical Psychology, 63(5): 718-725

Toward a Sociology of the Network Society

jstengel — Mon, 09 Feb 2009 16:30:00 GMT

This article invokes a provocative view of a changed society poised on the cusp of high potential for innovation because all of us (in developed nations) now function amidst a vast complex of macro-level social networks (and thusly a new framework for the study of sociology needs to be erected). Just when consortium members might think they’re ready to spin hypotheses about promising innovative inroads in technology and organizational culture, Castell comes along with fresh insight into our changed society in ways that possibly alter the rules of engagement in innovation generation.

The author offers a glimpse of some macro-level social and technological networks that amend our current view of society enough to seemingly need to be accounted for in present thinking (yet give sufficient pause to hamper forward progress?).

The author cites information technology, globalization, the WWW, and the dilution of the sovereign nation-state as four axes of significant societal restructuring we’ve undergone. He further describes examples of macro-level networks impacting our lives in ways that have indirectly reshaped society as we’ve known it; these changes brought on by current technology, the author seems to suggest, have spawned a diffusion of innovative “side effects” on the social structure itself.

Citation

Castells, M. (2000). "Toward a Sociology of the Network Society." Contemporary Sociology 29(5): 693-699.

Technology Transfer and Public Policy: A Review of Research and Theory

jstengel — Fri, 30 Jan 2009 16:20:17 GMT

The author provides an exhaustive review of technology transfer/diffusion through his own lens: i.e., a framework he calls the Contingent Effectiveness Model; it earns its name from the fact that there are multiple ways of defining transfer effectiveness (6 by Bozeman’s count). Before outlining the key factors in parsing out what constitutes innovation/technology transfer/diffusion, it should be noted that (1) Bozeman’s focus is mainly on public sector (university and government labs), and very little regarding private sector technology transfer research; and (2) it clearly represents “physical technology and its end products” more than, say, social science/technology and its “end products.”

Regardless, from both “hard” and “soft” technology vantage points, Bozeman’s article (and its inclusive model) is instrumental for any entity embarking on a technology/innovation enterprise; especially in terms of articulating with great clarity what the stakeholders are and are not proposing in very purposeful terms. Early in the paper, Bozeman depicts the anatomy of technology transfer with surgical precision by: (a) defining technology, (b) demarcating the technology object to be transferred, (c) giving insight to the natural instability of technology (which is always evolving), and (d) defining technology transfer itself.

Bozeman’s six effectiveness categories for demonstrating use of any technology-transfer object’s use include: (1) Opportunity Cost; 92) Scientific & Technical Human Capital; (3) Political; (4) Economic Development; (5) Market Impact; and (6) “Out-the-Door”. His model (and all supporting research cited) devolves into five interrelated factors which influence transfer effectiveness. Each of the five factors has its own unique set of characteristics (extrapolated by research reviews): (1) Transfer Agent; (2) Transfer Media; (3) Transfer Object; (4) Demand Environment; and (5) Transfer Recipient. Several explanatory tables support Bozeman’s model’s contentions with examples and highlighted focal points. Thus, this paper serves two objectives, as a relatively thorough review of supportive research on the one hand, and as a useful lens or filter for logic modeling of any venture ambitiously taking to task technology/innovation transfer. transfers from business service sectors to other industrial sectors. The RISE project deals with the roles of knowledge-intensive economic activities and of research and technology organizations within innovation systems. It stresses the importance of an anthropological interpretation of the role of services, service providers and the relationships between them. The INNOCULT project looked at National innovation systems, with a focus on institutional innovation. It stressed the role of cultural factors in national innovation systems, the role of culture in research and innovation networks, and the effect of internationalization on national innovation systems. UNIREG details the various roles of universities in producing and transmitting knowledge, as cultural agents, and as leading or facilitating agents in the regional and local governance systems. The KISINN project analyzes innovation processes at the institutional, spatial, organizational and strategic levels, and emphasizes the accumulation and circulation of knowledge and competence. Particular attention is devoted to the analysis of concrete; learning by interacting; processes in different countries, regions and sectors.

Citation

Bozeman, B.(2000). Technology Transfer and Public Policy: A Review of Research and Theory. Research Policy, 29: 627-655.

“Futures That Work: Using Search Conferences to Revitalize Companies, Communities, and Organizations”

jstengel — Wed, 28 Jan 2009 16:13:21 GMT

A search conference is a participative planning event that enables people to create a plan for the most desirable future of their community or organization, a plan they carry out themselves. Search conferencing has a long, rich history. The first search conference happened in 1960 in Great Britain when leaders of two aircraft engine manufacturers came together in the first search conference to merge their two companies into one. The inventors of the method and the leaders of that first search were social scientists Fred Emery and Eric Trist. The search conference is a practical way to build communities of people who step up to the challenges of our turbulent times and take responsibility for making change happen in a purposeful way. As the world becomes more and more turbulent, the need is great for people to form communities to search for their desirable futures together. The search conference puts people in the driver’s seat of change, so they can steer together toward the future they want for their system, making adjustments as they go forward.

This book has a detailed description of what happens in a search conference, identifies the many ways it can be used, the principles that it is based on, how to plan for a search conference, and stories from many different organizations, from a Palestinian YMCA to Microsoft. It is a good guide for those who want to bring together a group of people to plan strategically and holistically and, importantly, in an accountable and sustainable way to implement what they plan. It also begins to shift organizations to a more open and participative culture.

Citation

Rehm, R., Cebula, N., Ryan, F., Large, M. (2002), “Futures That Work: Using Search Conferences to Revitalize Companies, Communities, and Organizations”, New Society Publishing and Hawthorn Press.

Innovation. Management Benchmark Study

jstengel — Mon, 26 Jan 2009 16:10:51 GMT

In the chapter, “Innovation,” Baker provides an overview of innovation research, definitions of the three main types of innovation (process, product/service, and strategy/business concept), the levels of innovation (incremental to radical), and capacities for innovation, which either enable or hinder organizations’ ability to embrace change. Process innovation involves an organization’s ability to respond to internal and external changes. Product service innovation concerns product and process innovation that occurs despite any evidence of necessity for change. Finally, strategy/business concept change attempts to radically and “proactively” respond to environmental and market demands with an eye to any future challenges before they exist.

The levels of innovation range from incremental change, which involves small adjustments to existing products and services as determined by internal or external forces, to radical innovation, which completely transforms either the marketplace or the economy by developing completely new products or services. Leveling provides a measure of the “newness” of innovations. As well as levels of innovation, Baker considers the impact of innovation. Impact can be measured, according to Christensen (1997), as either sustaining or discontinuous.

Baker identifies the drivers of innovation. They include:

Financial pressures
Competition
Shelf life of a product or service
Changes in values
Regulations
The need for sustainable development
Accountability
Social consciousness
Demographics
Rising expectations
New technologies
Shifting economies

Factors which can present obstacles to innovation include the following:

Individuals
The project itself
The organization
The environment

An organization’s ability to successfully transfer innovation depends on its absorptive capacity (Cohen and Levinthatl 1990). Absorptive capacity, or “organization as sponge explanation,” refers to the organization’s ability to absorb input and “squeeze out” results or outcomes (Fiol 1996). The impact of innovation is directly related to an organization’s ability to “absorb” new information and knowledge and, therefore, produce completely new ideas, services, or products.

Baker identifies four levels of innovative capacity: individual, project, organizational, and environmental. Citing Hamel (2000), Baker enumerates internal and external structures an organization must create in order to achieve innovation competency:

Have a fluid organizational boundaries and an open market for talent
Transform organizational strategy - - both from the top-down and from the bottom- up
Create an open market for capital investment and rewards
Manage the risk
Create a culture and a structure that promotes innovation

The author has developed a list of questions by which an organization or company can assess its innovativeness. This check list includes items covering the “newness” or originality of the innovation, the involvement of the workforce, external influences, the level of application, and the capacity of the organization for innovation.

Finally, Baker reinforces the theory that successful, transferable innovation needs to be understood from an organizational perspective and that “keeping up with innovations” is no longer enough to guarantee success. Organizations need to be innovation leaders from a structural standpoint, not only a project/product view.

Tools/Knowledge Objects/Resources/Contacts/Etc:

Resources: Biech, E. 1996. Creativity and Innovation: The ASTD Trainer’s Sourcebook. New York: MacGraw-Hill.
Christensen, C.M. 1997. The Innovator’s Dilemma. Boston: Harvard Business School Press.
Cohe, W.M. and D.A. Levinthal. 1990. “Absorptive Capacity: A New Perspective on Learning and Innovation. Administrative Science Quarterly 35:128-152.
Fiol, C. Marlene. 1996. “Squeezing Harder Doesn’t Always Work: Continuing the Search for Consistency in Innovation Research. Academy of Management Science 21(4):1012-1021.
Hamel, Gary. 2000. Leading the Revolution. Boston: Harvard Business School Press.

Citation

Baker, K. (2002). Innovation. Management Benchmark Study. Washington, D.C., Department of Energy Office of Science, Air University (USAF) 16.

"Counselor perceptions of organizational factors and innovations..." by Joe, G. et al.

Donna Cooper — Fri, 22 Feb 2008 00:45:00 GMT

Researchers from Texas Christian University studied a large population of drug abuse counselors focusing on the individuals as treatment counselors with respect to training and organization, a departure from the norm of studying counselors in the aggregate to ascertain the behavior of the organization.

[Joe, G. W., Broome, K. M., Simpson, D., & Rowan-Szal, G. A. (2007). Counselor perceptions of organizational factors and innovations training experiences. Journal of Substance Abuse Treatment, 33 (2), 171-182.]

Three primary questions were considered:

How do counselors' perceptions of themselves and their working environment relate to readiness for and responsivity to innovations training?
How do they rate their previous experiences in adopting workshop training materials?
Will patterns (types) of counselors be predicted and identified from organizational climate perceptions?

The primary assessment tool utilized for data collection was the TCU-developed Organizational Readiness for Change (ORC), a 129-item assessment measuring 18 categories across 4 primary areas: Program needs/pressures (indicative of training motivation); program resources; staff attributes; and organizational climate (Lehman, Greener and Simpson, 2002).

Hypotheses

The ORC scales have been proven to be significant predictors of innovation at the program level (Simpson, et al, 2007), treatment program process indicators (Greener, et al., 2007), and predictors of attitudes toward evidence-based practice (EBP) at the individual level (Saldana et al. 2007).

Therefore, it was hypothesized that:

As measured by the ORC, several patterns or types of counselors could be described from their perceptions of organizational climate and perceived job performance abilities;
These descriptive patterns should be applicable across a variety of treatment settings;
Counselors who have more "favorable" profiles should be more engaged in efforts to improve work performance, such as innovative training opportunities.

Research Design Method

Treatment programs were recruited by four Addiction Technology Transfer Center networks (ATTC) sponsored by the Substance Abuse and Mental Health Services Administration. Data collection and administration procedures were approved by the Institutional Review Board at TCU. Participation was solicited by contacting treatment programs which provided mailing lists and letters of support encouraging participation in subsequent ATTC-sponsored workshop trainings. Counseling staff completed the ORC assessment 3 months prior to the training conferences. An average of three assessments was completed per treatment unit.

Confidentiality was ensured by providing participants postage-paid, pre-addressed envelops that could be sealed and mailed directly to TCU. One state offered electronic collection of the ORC via the internet. Overall, the paper version of the ORC was administered to counselors in 180 programs, while the electronic version was administered in 165 programs.

Sample Population

A total of 1,047 counselors voluntarily participated in the assessment in 10 states.
Data was collected between 6/2000 and 6/2003.
35% of the participants were male;
Average age was 45;
71% were Caucasian, 17% African American, and 12% were Hispanic.
60% were certified counselors, 12% working to receive certification, 2% previously certified, and 26% not certified.
Counseling experience varied -- 16% less than one year, 20% 1-3 years, 14% 3-5 years, and 49% more than 5 years.

Measures and Reliability

The staff attributes and organizational climate domains of the ORC comprised the profile of measures identifying the latent classes of counselors.

Staff attributes addressed growth (5 items with a coefficient α = .68); efficacy (5 items with a coefficient α = .68); influence (6 items = with a coefficient α = .64).
Organizational climate addressed clarity of mission and goals (5 items with a coefficient α = .74); staff cohesiveness (6 items with a coefficient α = .88); staff autonomy (5 items with a coefficient α = .56); open communication (5 items with a coefficient α = .83); stress (4 items with coefficient α = .82); and openness to change (5 items with coefficient α = .73).

The other two ORC domains - needs and pressures, and resources - were used as predictors of the counselor groups.

Needs and pressures addressed program need for improvement (8 items, coefficient α = .90); immediate training needs (8 items, coefficient α = .88); and pressures for change (7 items, coefficient α = .68).
Resources addressed offices (4 items, coefficient α = .74); staffing (6 items, coefficient α = .74); training resources (4 items, coefficient α = .63); equipment (computer access with 7 items, coefficient α = .77); and internet (electronic communication with 4 items, coefficient α = .78).

There were 14 questions on the ORC which addressed previous workshop training experiences as related to the expectation that there would be a correlation between counselor job-related efficacy/climate and job engagement.

Training exposure consisted of 5 items with a coefficient α of .62;
Individual adoption (pertaining to prior workshops) consisted of 4 items with a coefficient α of .78;
Program adoption (pertaining to agency wide use of prior workshop training) consisted of 3 items with a coefficient α = .81;
Training satisfaction (pertaining to prior year trainings and training opportunities in general) consisted of 2 items with a coefficient α = .77).

Analyses

Latent profile analysis (LPA) of staff attributes and organizational climate measures were conducted using MPlus software. As counselors were nested within clinics, a bi-level model for estimating patterns with emphasis on within-program counselor differences was used. The advantage of the LPA, the authors note, is that it has the ability to address incomplete data, using information from all cases (complete or incomplete), and assuming that missing information is unrelated to the values that are missing.

The researchers based their decisions about the appropriate number of latent classes on the Bayesian information criterion (BIC), adjusted for sample size and entropy index (Muthen & Muthen, 2004).

Multinomial logistic regressions of the latent grouping variable on program resources, program motivation and counselor background were performed in a second set of analyses, again with the MPlus software. This was done to identify key predictors of the latent classification patterns.

A third set of analyses using 2 models in a chi-square difference test, addressed the hypothesis that the latent groups were significantly different regarding innovations training and adoption.

Limitations

The following limitations may inhibit the researchers' ability to generalize findings across a wide array of treatment programs:

counselors and programs represent a convenience sample gathered from a large number of treatment programs covered by the ATTC;
the data are cross-sectional in nature;
some participating programs were motivated by the innovation training offered.

Findings

Three classes of counselor perceptions of organizational dynamics in work environments were discovered and labeled:

isolated (43%),
integrated (43%), and
exceptional (14%).

The ‘isolated' group was identified as having higher stress and also had lower cohesiveness and communication scores. The ‘integrated' group had a positive view of the programs' climates and moderate stress. The ‘exceptional' group showed low stress and high scores for other indicators, with cohesiveness the highest.

Considering the combined percentages (57%) for integrated and exceptional classes, a majority of the counselors had positive perceptions of their work environment. Also, a majority of the counselors feel confident about their job efficacy and adaptability (scores of near or above 40 out of a 50-point scale).

The level of organizational functioning in the work environment was found to be related significantly to various training-experience measures.

The source of strongest discrimination came from the program resources domain. All of these scales showed that those perceiving better offices, program staffing, training resources, equipment, and internet were more likely to be in the two higher treatment organizational dynamics work environment groups when compared to the isolated group.

The finding that experienced counselors were less likely to report integrated and supportive environments, a valuable ingredient for adopting new techniques, raises "special concerns" that should be studied further, note the authors.

Summary Review

The present research establishes the relationship of organizational climate as an aid in identifying whether counselors may be ready to engage in the process of adoption of innovations.

The results emphasize the importance of communication, staff cohesiveness, clarity of mission, and work stress, which are key components of the organizational dynamics of the treatment programs.

Tools/Knowledge Objects/Resources

Greener, J. M., Joe, G. W., Simpson, D., Rowan-Szal, G. A., & Lehman, W. E. K. (2007). Influence of organizational functioning on client engagement in treatment. Journal of Substance Abuse Treatment, 33(2), 139-147.
Lehman, W. E. K., Greener, J. M., & Simpson, D. D. (2002). Assessing organizational readiness for change. Journal of Substance Abuse Treatment, 22(4), 197-209.
Muthen, L. K., & Muthen, B. O. (2004). Mplus user's guide: Statistical analysis with latent variables. Los Angeles: Muthen & Muthen.
Saldana, L., Chapman, J. E., Henggeler, S. W., & Rowland, M. D. (2007). The Organizational Readiness for Change scale in adolescent programs: Criterion validity. Journal of Substance Abuse Treatment, 33(2), 159-169.
Simpson, D. D., & Flynn, P. M. (2007). Moving innovations into treatment: A stage-based approach to program change. Journal of Substance Abuse Treatment, 33(2), 111-120.

"Organizational readiness for change and opinions toward treatment..." by Fuller, B. E. et al.

Donna Cooper — Thu, 21 Feb 2008 00:01:00 GMT

The authors noted a significant gap between evidence-based practice (EBP) innovations and their adoption in the mental health care profession.

[Fuller, B. E., Rieckmann, T., Nunes, E. V., Miller, M., Arfken, C., Edmundson, E., et al. (2007). Organizational readiness for change and opinions toward treatment innovations. Journal of Substance Abuse Treatment, 33(2), 183-192.]

Based on an in-depth review of implementation research, six core elements were identified as critical to adoption of and maintaining EBP programs:

selection of staff capable of implementing such programs;
staff training;
on-going coaching;
supervision and assessment measures to provide staff performance feedback;
program evaluation;
fostering of implementation/maintenance via administrative supports (Fixsen, Naoom, Blasé, Friedman & Wallace, 2005).

They assessed the utility of the Organizational Readiness for Change (ORC) scale (Lehman, Greener & Simpson, 2002) using a regression model on four types of EBP:

manuals,
medication,
integrated mental health services, and
motivational enhancements. -

They also provided a literature review of these four EBP types.

Research Design Method

Three interrelated self-report surveys focusing on organizational, treatment unit, and workforce issues were utilized and completed sequentially. Opinion items were measured on a five-point Likert scale. Complete details on collection methods are provided in other literature (McCarty, Fuller, Arfen, Miller, Nunes, Edmundson, et al, 2007). Surveys were completed online or via paper transmission. The workforce survey was distributed via sealed correspondence to specified employees, and, most often, distributed and completed during staff meetings.

The Oregon Health & Science University Institutional Review Board (IRB) reviewed and approved study procedures, and 40 IRBs reviewed and approved the protocol. Respondent participation could be declined.

Items for the organizational self-report survey included information on ownership status, primary service setting, and company size. Response rate was 95%. Information in the treatment unit survey covered care and service levels, facility accreditation and licensure, patient characteristics, revenue sources, staffing, staff retention, and scales assessing treatment environment and philosophy. A response rate of 91% was reported.

The workforce survey assessed such items as training, credentials, education, years of experience, and opinions toward specific treatment types. It also included the 115-item ORC scale (Lehman, Greener & Simpson, 2002) which assesses factors influencing organizational change. Four categories covering 18 topics were addressed:

Motivation for change based on three factors -- program need for improvement, perceived training needs and pressure to change;
institutional resources measured five areas covering adequacy of resources - office space, staffing, training resources, computer access; and e-mail/Internet useage;
staff attributes influencing organizational change - professional growth potential, counseling efficacy, influence on co-workers, and adaptability;
organizational climate, which included six topics -- agency mission and goal clarity; staff cohesion; staff autonomy; management openness to staff communication; perception of stress level; and openness to change.

A response rate of 71% was reported for the workforce survey.

An ORC scale initial validation study indicated that the 18 topics were generally single factors and that the psychometric properties were strong (Lehman, et al., 2002).

Sample Population

The National Drug Abuse Treatment Clinical Trials Network (CTN) includes 17 regional research centers and approximately 150 community-based groups which provide AOD treatment services, and whose membership fluctuates over time as protocols dictate. The network was founded to provide support for innovations in EBP and evidence about efficacy of those practices. The sample population included:

participants from 249 treatment units in this network including
outpatient treatment programs (n = 79)
residential and inpatient programs (n = 95)
detoxification programs (n = 34) and
methadone programs (n = 41)
Treatment units with at least five staff members were required to allow for the use of models using group mean centering
Respondents included administrators and staff

Measurements

Factor analysis was used on opinion items to form factor scores. Hierarchical linear modeling assessed the influence of organizational characteristics and ORC domains on factor scores. (Comprehensive details of the methods and results are provided in the full journal article.) Five factor scores were generated and modeled as the outcome variable in a bi-level random effects regression model. The five score categories are:

EBP and manualized treatment;
Confrontation and non-compliance discharge;
medications;
mental health services;
motivational incentives.

Level 1 predictors were the ORC scales centered on the group mean to note within-group variation. Level 2 predictors included eight organizational variables: 1) independent treatment centers; 2) for-profit status; 3) annual revenue; 4) national accreditation; 5) residential treatment centers; 6) outpatient treatment services; 7) methadone services; 8) detoxification centers. The algebraic framework of the random-effects mixed model is provided in this article.

Findings

A large and heterogeneous population for assessing the influence of the ORC scales in EBP was provided by the CTN.
Staff attributes appeared to have positive and negative effects with regard to EBP.
Within the ORC domains, perceived need for improvement was key to motivation for change, and staff members scoring high here also endorsed treatment manuals, medications and motivational incentives.
Staff internet access surfaced as a significant institutional resource.
Staff who rated internet access as adequate showed more support for treatment manuals, medications and motivational incentives and less support for confrontation and non-compliance discharge.
Staff who rated themselves as more influential with peers indicated more support of manuals, integrated mental health services and motivational incentives.
Staff indicating less influence with peers appeared more supportive of confrontation and non-compliance discharge.
Perception of opportunities for professional growth affected positively willingness to use medications, and negatively influenced the use of confrontation and non-compliance discharge.

Organizational climate measures were also rated influential:

Perceived stress ratings were lower when support for using confrontation and non-compliance discharges was evidenced.
Stress ratings were higher with more support for integrated mental health services - apparently reflecting change and integration pressures.
A strong sense of mission clarity was also associated with use of treatment manuals and integrated mental health services.

The authors note that many of the 13,000+ specialty treatment centers for drug and alcohol abuse in the U.S. are under pressure to improve economic efficiency while providing EBP-proven treatment. Analysis of organizational and workforce variables added to the validity of the ORC, and indicate that the ORC measures important factors in organizational readiness for change. The analysis of the ORC dimensions within the CTN indicates that treatment center leaders should systematically involve their staff in the change process. The authors recommend subsequent examination of ORC influence on patient outcomes and program performance in clinical trials.

Tools/Knowledge Objects/Resources/Contacts:

Lehman, W. E. K., Greener, J. M., & Simpson, D. D. (2002). Assessing organizational readiness for change. Journal of Substance Abuse Treatment, 22, 197-209.
McCarty, D., Fuller, B., Arfken, C. L., Miller, M., Nunes, E. V., & Edmundson, E. (2007). Direct care workers in the National Drug Abuse Treatment Clinical Trials Network: Characteristics, opinion, and beliefs. Psychiatric Services, 58, 181-190.

"Assessing program needs and planning change" by Rowan-Szal, G. A. et al.

Donna Cooper — Wed, 20 Feb 2008 21:54:00 GMT

Personnel from the Texas Christian University (TCU) examined the organizational functioning of alcohol and drug abuse treatment centers across two states in the Gulf Coast with reference to strategic planning, in particular staff training needs and transfer of evidence-based practice into clinical settings.

[Rowan-Szal, G. A., Greener, J. M., Joe, G. W., & Simpson, D. D. (2007). Assessing program needs and planning change. Journal of Substance Abuse Treatment, 33(2), 121-129.]

Two TCU-developed assessments were used for their evaluation:

The Organizational Readiness for Change (ORC) assessment, which evaluates organizational traits -- needs and pressures, resources, staff characteristics, and organizational environment -- that predict readiness for change;
The Program Training Needs (PTN) assessment helps identify innovative treatments based on staff input from seven areas -- staff training needs, training content and training strategy preferences, training barriers and satisfaction with training, program facilities/climate and computer resources. The PTN was created as a complementary assessment to the ORC to provide staff with more detailed information regarding treatment issues than that provided by the ORC.

Problem Studied

Their quantitative analysis had three goals:

To determine the basic psychometric properties of the seven scales in the PTN, its factor and internal structure (reliability and dimensionality), and to explore relationships among its seven scales;
To ascertain differences and similarities between perceptions of training needs as reported by clinical supervisors and staff.
To explore training needs identified by the PTN assessment within a more "global" context and with relation to the ORC assessment.

Sample Population

PTN assessment data was collected from 589 counselors at 194 treatment centers (members of the Gulf Coast Addiction Technology Transfer Center) in two states.
A total of 579 counselors completed the ORC assessments, which were collected from 231 treatment programs in both states.
State A had 192 completed PTN surveys (33% of the data sent) while State B had 397 completed (67%).
There were 174 ORC assessments collected (30%) from State A and 405 ORC assessments (70%) collected from State B.
Caucasians represented 63% of respondents, African Americans 24%, Hispanics 14%.
Sixty-nine percent of all respondents were female.
Regarding years of counseling experience, 45% of the participants had at least 5 years experience in drug and alcohol counseling; 21% held their current jobs for at least 5 years and 43% of respondents had been employed in their current position for less than one year.
Clinical supervisors accounted for 14% of PTN completion rate while 86% of the assessments were completed by counseling staff (81 and 502 PTNs, respectively).

Research Design Methods

Quantitative data collection from both PTN and ORC surveys was conducted over a two-year period spanning 2003 and 2004. The TCU Institutional Review Board approved all protocols used, and funding was provided by the National Institute on Drug Abuse.

State A: Fifty-nine treatment units were sent self-addressed, stamped envelopes for voluntary survey return. For PTN protocols, there was a 59% participation rate, or 192 responses out of 330 forms sent. The ORC return rate was 52%, with 174 out of 330 assessments returned to TCU researchers. The PTN assessment forms were sent four months after the mailing of the ORC scales.

State B: Researchers used PsychData to provide online ORC assessments in June 2003, and PTN surveys were conducted via the internet in November and December 2003. PTN assessments were sent to 135 programs within the state, and ORC forms were sent to 173 treatment centers. TCU researchers said they were unable to calculate response rates for these assessments from State B because they were online responses.

Due to the researchers' need to restrict information to that data which was linked between the PTN and ORC surveys, the final sample was reduced to one-third of that originally collected to 195 PTN assessments and 174 ORC assessments from a total of 97 matched treatment centers from both states.

Assessments

The seven PTN areas covering 54 items are identified as follows:

Program facilities & climate - includes 7 items assessing the physical office space available and the overall treatment staff morale;
Program computer resources - includes 5 items covering the use of and adequacy of computer equipment including e-mail and internet access;
Staff training needs - incorporates 10 items assessing the skills such as cognitive and behavioral interventions, assessments and monitoring techniques which counselors think would help them perform their jobs better;
Training content preferences - (8 items) assesses perceptions of need for certain types of training such as diagnostic training tools, dual diagnosis and special populations;
Preferences for training strategy - (10 items) centers on types of training that staff prefer to receive along with logistics, such as full-day training, onsite consultation and using conceptual models;
Training roadblocks - (10 items) on staff thoughts about barriers to training such as budget constraints, applying ideas from workshops, and workload pressures that could interfere with a counselor's ability to attend training events;
Training satisfaction - (4 items) reports on the satisfaction level of staff regarding the sources, quality and types of training offered during the year.

Scoring for the PTN was as follows: 10=strongly disagree; 20=disagree; 30=undecided; 40=agree; 50=strongly agree, with mean scores higher than 30 indicating overall agreement.

For a description of the ORC's psychometric properties, scales and rationale for the assessment, please refer to Lehman et al. (2002). The ORC covered 115 items representing 18 subscales spanning the four aforementioned organizational traits of needs and pressures, resources, staff characteristics, and organizational environment. Responses were graded on a five point scale (disagree strongly, disagree, uncertain, agree, agree strongly) with scores being reverse-scored computing the mean score and multiplying by 10. Higher scores represented more agreement while lower scores represented levels of disagreement, while 30 represented a neutral score.

ORC composite scores were computed for each of the four domains using the 18 domain subscales:

The Needs and Pressures Index (NPI: motivation for treatment) is the mean value of the scores on the 3 needs/pressures subscales - training needs; program needs and pressures for change;
The Organizational Climate Index (OCI), a composite measure, is the mean value of scores on 6 subscales - mission, staff cohesiveness, staff autonomy, communication, stress and openness to change;
The Institutional Resources Index (IRI) is the mean value of scores on 5 subscales - offices, staffing, training resources, equipment and internet;
The Staff Attributes Index (SAI) is the mean value of the scores on the 4 staff characteristics subscales - influence, adaptability, efficacy and growth.

Measurements

Confirmatory factor analysis (CFA) using Proc CALIS (SAS, 2003) software evaluated the fit of the 54 PTN items to the hypothesized scales. Cronbach's α was used to compute reliability of PTN scales, and the relationships among seven PTN scale composites were examined using Pearson's correlations.

Comparisons between clinical supervisors and staff responses were made by using analysis of variance to examine mean score differences between the two groups: Staff responses in a treatment unit were averaged and compared to supervisor's responses from the same unit.

The relationship between the PTN scales and the ORC's organizational functioning scales were assessed based on analyses at the treatment unit level. Pearson's correlations were used to evaluate the relationships between the PTN and ORC domain indexes.

Findings

For the seven-factor CFA of the 54 PTN items, surveyed by the PTN, the RMR (root mean square residual) was .065 while the RMSEA (root mean square error of approximation) was .06. which suggests that the seven factors are "appropriate" for describing the PTN, but not a good fit. Browne and Cudeck (1993) note that an RMSEA of .05 or less indicates a close fit in relation to degrees of freedom, while values up to .08 indicates a fit "having reasonable errors of approximation in the population." Hu and Bentler (1999) present evidence that RMSEA scores of approximately .06 and SRMR (standard root mean square residual) results of close to .08 indicate a relatively good fit between a hypothesized model and the observed data.

Loadings from the CFA indicate that all items in the Program facilities and climate, Training satisfaction, Staff training needs and Training barriers would be "marker variables for the scales because all loadings are higher than .40. However, some of the items for the Training content and Training strategy preferences along with Program computer resources did not meet that cutoff, which explains why the fit was not better.

The reliability of the PTN scales was computed using Cronbach's α. Except for the Program Computer Resources scale, the scales had a reliability of .70 or higher ranging from .71 to .92.

These psychometric analyses indicate that the PTN adequately fits the pre-conceptualized seven-factor structure and that its scales have acceptable reliabilities.

The differences and similarities between the perceptions of supervisors and subordinate staff members revealed significant differences in only one area of the PTN survey, Satisfaction with Training. Clinical supervisors rated outside training and state funded drug and alcohol agencies significantly higher than did the staff. No other significant differences were reported.

Finally, the relationship between ORC NPI composite index was shown to be positively related to two PTN areas, Staff Training and Preferences for Training. This indicates that agencies which reported higher training needs and preferences for specific training content also received greater needs and pressures reflecting motivation for organizational change. Results also showed that treatment units with higher PTN scores for Program Facilities and climate scale had higher ORC scores for those corresponding measures. Importantly, those centers with the greatest number of resources and most favorable climate scores also, perhaps not surprisingly, reported fewer barriers to success and greater satisfaction with training strategies and opportunities.

Implications

It has been suggested (Backer, 1993; Peterson, 1998; Goldstein, 1991, 2002) that any attempt to initiate organizational change and successfully transfer innovation into evidence-based practice must be preceded by an organizational sense of urgency based on a needs assessment of both the current organizational structure and the needs and skills of the individuals working within that organization. The researchers involved in this survey and assessment of PTN and ORC surely subscribe to that paradigm.

In an effort to successfully transfer evidence-based treatment programs into the everyday practice of hundreds of drug and alcohol counselors and supervisors, Rowan-Szal, et al., in conjunction with TCU, have built a template of indexes designed to measure both the needs of specific counseling centers and their openness to accepting organizational change. This "systematic assessment of needs and staff readiness for innovations" is necessary before any program change or knowledge acquisition can occur. Organizations which have had the opportunity to identify their own particular challenges and successes are, according the authors, much more likely to adapt systematic reformation, based on research, and initiate those reforms over time.

Tools/Knowledge Objects/Resources/Contacts:

Backer, T. E. (1993). Information alchemy: Transforming information through knowledge utilization. Journal of the American Society for Information Science, 44, 217-221.
Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136-162). Newbury Park, CA: Sage.
Goldstein, I. L. (1991). Training in work organizations. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (2nd ed.). Palo Alto, CA: Consulting Psychologists Press.
Goldstein, I. L., & Ford, J. K. (2002). Training in organizations: Needs assessment, development, and evaluation (4th ed.). Belmont, CA: Wadsworth.
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.
Lehman, W. E. K., Greener, J. M., & Simpson, D. D. (2002). Assessing organizational readiness for change. Journal of Substance Abuse Treatment, 22, 197-209.
Peterson, R. (1998). Training needs assessment: Meeting the training needs for quality performance. Sterling, VA: Kogan Page.
SAS Institute. (2003). SAS/STAT user's guide. Cary, NC: SAS Institute.